AmiGO2 install

GOLR install:

There are a few prerequisites to getting GOLR installed. They appear to be Maven and OwlTools. To install maven, I followed the instructions from here: http://blog.gluster.org/2013/08/yum-install-maven-yes-you-can/, seemed to work without error.

OWLTools

Then, to install OWLTools, I did the following:

cd /lemma/justin
svn co http://owltools.googlecode.com/svn/trunk/ owltools

This creates the owltools directory in my space on lemma.

To compile OWLTools, run:

cd OWLTools-Parent
mvn clean package

This will take about half an hour.

SOLR

Grab the 3.6 version of SOLR from apache and unpack:

wget http://archive.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
tar zxvf apache-solr-3.6.2.tgz

Pull down the latest version of AmiGO:

cd /data/www/planteome_dev
git clone https://github.com/geneontology/amigo.git amigo

Go back to the SOLR directory and replace the config with those from GOLR:

cd /lemma/justin/SOLR/apache-solr-3.6.2/example/solr/conf
cp schema.xml schema.xml.org
cp solrconfig.xml solrconfig.xml.org
cp /data/www/planteome_dev/amigo/golr/solr/conf/s* ./

I don't see any edits that need to be made at this point. I think when we move to additional ontologies, we will have to modify schema.xml to account for new relationship types. Cross that bridge when we get there.

Start SOLR(GOLR):

cd ../../ (be in the example directory)
java -jar start.jar

Now that it seems to be starting up fine, daemonise it so that it runs at a service at boot (Note that any service needs to be okayed by Chris before started):

Add the init.d script as outlined on this page

Had to modify that script as follows to make it work on CentOS:

elserj@palea init.d]$ diff solr solr.org 
16,19d15
< # Source function library.
< [ -f /etc/rc.d/init.d/functions ] || exit 0
< . /etc/rc.d/init.d/functions
< 
24c20
<     daemon "cd /data/www/planteome_dev/SOLR/example; java -jar start.jar &> /var/log/solr/solr.log &"
---
>     daemon --chdir='/data/www/plantome_dev/example' --command "java -jar start.jar" --respawn --output=/var/log/solr/solr.log --name=solr --verbose
40c36
<     ps -aef | grep start.jar | grep -v grep | awk '{print $2}' | xargs -i++ kill -9 ++
---
>     daemon --stop --name=solr  --verbose

At this point, SOLR is now running. Next is to load data to it.

Load data into GOLR instance

First we need to pull down the latest AmiGO code from git:

cd /data/www/planteome_dev
git clone https://github.com/geneontology/amigo.git amigo

In the metadata folder are the yaml files that tell owltools how to populate the fields in GOLR

Edit the golr/Makefile to fit our server:

[elserj@palea golr]$ diff Makefile Makefile.org 
18c18
< MAVEN_EXE ?= /usr/bin/mvn
---
> MAVEN_EXE ?= ~/local/src/java/apache-maven-3.0.4/bin/mvn
21c21
< SOLR_URL ?= http://dev.planteome.org:8983/solr 
---
> SOLR_URL ?= http://localhost:8080/solr/ 
24,27c24,27
< OWLTOOLS_ROOT ?= /nfs0/BPP/Jaiswal_Lab/justin/owltools/
< BBOP_JS_ROOT ?= /data/www/planteome_dev/amigo/external/
< #PANTHER_FILES_DIR ?= ~/local/src/svn/geneontology.org/trunk/experimental/trees/panther_data/
< #SOLR_DATA_ROOT ?= /srv/solr
---
> OWLTOOLS_ROOT ?= ~/local/src/git/owltools/
> BBOP_JS_ROOT ?= ~/local/src/git/bbop-js/
> PANTHER_FILES_DIR ?= ~/local/src/svn/geneontology.org/trunk/experimental/trees/panther_data/
> SOLR_DATA_ROOT ?= /srv/solr

load-ontology

Try to run the load-ontology part of the Makefile:

make load-ontology

This will take some time as it appears to be updating the owltools and then loading the ontologies

At this point, I get an error when trying to load the ontologies. Working with Seth to figure it out.

Errors were mostly caused by memory issues. Had to comment out the ncbitaxon.owl ontology from the list of ones ran. Upped the memory to 20GB on palea. May eventually have to look at running the load on one of the cluster nodes as they have much more RAM. Would have to figure out how to open up the SOLR port to the internal network though.

Now to get annotation test data into GOLR

make load-test-gaf

This took a few minutes to run, not long.

At this point, data for an initial test is loaded into SOLR, next is to get AmiGO2 up and running off that instance.

AmiGO2 install

Since we already have pulled the amigo code from git, we have to work on configuring it:

cd amigo/conf
cp .initial_values.yaml amigo.yaml

Change the amigo.yaml file as follows:

[elserj@palea conf]$ diff amigo.yaml .initial_values.yaml 
5c5
<   value: http://planteome.org/amigo/cgi-bin/crop-amigo
---
>   value: http://amigo1.geneontology.org/cgi-bin/amigo
17c17
<   value: /data/www/planteome_dev/amigo/perl/bin
---
>   value: /home/sjcarbon/local/src/git/amigo/perl/bin
21c21
<   value: http://dev.planteome.org
---
>   value: http://localhost:9999
33c33
<   value: http://dev.planteome.org/solr/
---
>   value: http://localhost:8080/solr/
37c37
<   value: /data/www/planteome_dev/amigo
---
>   value: /home/sjcarbon/local/src/git/amigo
41c41
<   value: planteome-dev
---
>   value: localhost
45c45
<   value: /data/www/planteome_dev/amigo/static
---
>   value: /home/sjcarbon/local/src/git/amigo/static
49c49
<   value: http://dev.planteome.org/static
---
>   value: http://localhost:9999/static
53c53
<   value: 'all|[GPTE]O:[u0-9]{7}'
---
>   value: 'all|GO:[0-9]{7}'
65c65
<   value: /data/www/planteome_dev/amigo/golr_timestamp.log
---
>   value: /home/sjcarbon/local/src/git/amigo/golr_timestamp.log
69c69
<   value: /data/planteome_dev/amigo/external/bbop.js
---
>   value: /home/sjcarbon/local/src/git/amigo/external/bbop.js
73c73
<   value: /data/www/planteome_dev/amigo/metadata
---
>   value: /home/sjcarbon/local/src/git/amigo/metadata

Now to run

make install

This gives an error about a missing yui-compressor, so need to get that installed.

sudo mkdir /usr/local/share/yui-compressor
sudo chown elserj: /usr/local/share/yui-compressor
cd /usr/local/share/yui-compressor
wget https://github.com/downloads/yui/yuicompressor/yuicompressor-2.4.7.zip
unzip yuicompressor-2.4.7.zip

Create the following script in /usr/local/bin/yui-comprssor

#!/bin/sh
# Quick and dirty front-end script to the yui-compressor that emulates
# /usr/bin/yuicompressor script from Ubuntu release
#
JAVA_HOME=/usr/java/default
YUI_JAR=/usr/local/share/yui-compressor/yuicompressor-2.4.7/build/yuicompressor-2.4.7.jar
# Run YUI Compressor
/usr/bin/java -jar $YUI_JAR "$@"

Make it executable with

sudo chmod +x /usr/local/bin/yui-compressor

Next problem is that rhino gives an error message about "invalid option modules". Still communicating with Seth on solution. Will update when solution found.

Solution was to comment out the test and docs part of the install target in the Makefile. Rhino was apparently only used for unit testing, which we don't really need to do.

Configure Apache2

The Apache config file needs to be modified

[elserj@palea amigo]$ diff amigo2-apache-local.conf conf/amigo2-apache.conf
2,4c2,3
< <VirtualHost *:80>
<     ServerAdmin elserj@science.oregonstate.edu
<     ServerName dev.planteome.org
---
> <VirtualHost *:9999>
>     ServerAdmin sjcarbon@localhost
8c7
<     DocumentRoot /data/www/planteome_dev/amigo
---
>     DocumentRoot /home/sjcarbon/local/src/git/amigo
11c10
<     <Directory /data/www/planteome_dev/amigo/static>
---
>     <Directory /home/sjcarbon/local/src/git/amigo/static>
20c19
<       ScriptAlias /$binnm /data/www/planteome_dev/amigo/perl/bin/$binnm
---
>       ScriptAlias /$binnm /home/sjcarbon/local/src/git/amigo/perl/bin/$binnm
48,59c47,48
< 
<  ProxyRequests Off
<         <Proxy *>
<         Order deny,allow
<         Allow from 192.168.1.52
<         </Proxy>
< 
<     ProxyPass /solr http://localhost:8983/solr
<     ProxyPassReverse /solr http://localhost:8983/solr
< 
<     ErrorLog /data/www/planteome_dev/amigo/log/error.log
<     ScriptLog /data/www/planteome_dev/amigo/log/cgi.log
---
>     ErrorLog /var/log/apache2/error.log
>     ScriptLog /var/log/apache2/cgi.log
65c54
<     CustomLog /data/www/planteome_dev/amigo/log/access.log combined
---
>     CustomLog /var/log/apache2/access.log combined

For this to work, the mod_macro module for Apache must be installed. I found it as a downloadable rpm, so I installed it that way:

ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel6/x86_64/mod_macro-1.1.11-2.el6.pp.x86_64.rpm
sudo rpm -Uvih mod_macro-1.1.11-2.el6.pp.x86_64.rpm

The proxy settings are to make it so the Server is able to talk to the SOLR data.

I changed the /etc/httpd/conf/httpd.conf file to have the following line for the dev.planteome.org section

Include /data/www/planteome_dev/amigo/amigo2-apache-local.conf

Note that this cannot go in it's own VirtualHost section as it will error because of a VirtualHost inside a VirtualHost

doing the

make install

now works without error.

However, the server doesn't work when navigated to. It looks like a perl module is missing. Installed the following via cpan:

CGI::Application::Plugin::Forward

Error is now about "Type of arg 1 to keys must be hash" Will work on this error.

This error apparently is the result of the older version of perl that we have installed. Seth fixed this in git, so I did a git pull and got past it.

However, another perl module needed to be installed via cpan:

CGI::Application::Dispatch

At this point Amigo2 appears to be working.

This page will be updated as I progress.