AmiGO2 install
GOLR install:
There are a few prerequisites to getting GOLR installed. They appear to be Maven and OwlTools. To install maven, I followed the instructions from here: http://blog.gluster.org/2013/08/yum-install-maven-yes-you-can/, seemed to work without error.
OWLTools
Then, to install OWLTools, I did the following:
cd /lemma/justin svn co http://owltools.googlecode.com/svn/trunk/ owltools
This creates the owltools directory in my space on lemma.
To compile OWLTools, run:
mvn clean package
This will take about half an hour.
SOLR
Grab the 3.6 version of SOLR from apache and unpack:
wget http://archive.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz tar zxvf apache-solr-3.6.2.tgz
Pull down the latest version of AmiGO:
cd /data/www/planteome_dev git clone https://github.com/geneontology/amigo.git amigo
Go back to the SOLR directory and replace the config with those from GOLR:
cd /lemma/justin/SOLR/apache-solr-3.6.2/example/solr/conf cp schema.xml schema.xml.org cp solrconfig.xml solrconfig.xml.org cp /data/www/planteome_dev/amigo/golr/solr/conf/s* ./
I don't see any edits that need to be made at this point. I think when we move to additional ontologies, we will have to modify schema.xml to account for new relationship types. Cross that bridge when we get there.
Start SOLR(GOLR):
cd ../../ (be in the example directory) java -jar start.jar
Now that it seems to be starting up fine, daemonise it so that it runs at a service at boot (Note that any service needs to be okayed by Chris before started):
Add the init.d script as outlined on this page
Had to modify that script as follows to make it work on CentOS:
elserj@palea init.d]$ diff solr solr.org
16,19d15
< # Source function library.
< [ -f /etc/rc.d/init.d/functions ] || exit 0
< . /etc/rc.d/init.d/functions
<
24c20
< daemon "cd /data/www/planteome_dev/SOLR/example; java -jar start.jar &> /var/log/solr/solr.log &"
---
> daemon --chdir='/data/www/plantome_dev/example' --command "java -jar start.jar" --respawn --output=/var/log/solr/solr.log --name=solr --verbose
40c36
< ps -aef | grep start.jar | grep -v grep | awk '{print $2}' | xargs -i++ kill -9 ++
---
> daemon --stop --name=solr --verbose
At this point, SOLR is now running. Next is to load data to it.
Load data into GOLR instance
First we need to pull down the latest AmiGO code from git:
cd /data/www/planteome_dev git clone https://github.com/geneontology/amigo.git amigo
In the metadata folder are the yaml files that tell owltools how to populate the fields in GOLR
Edit the golr/Makefile to fit our server:
[elserj@palea golr]$ diff Makefile Makefile.org 18c18 < MAVEN_EXE ?= /usr/bin/mvn --- > MAVEN_EXE ?= ~/local/src/java/apache-maven-3.0.4/bin/mvn 21c21 < SOLR_URL ?= http://localhost:8983 --- > SOLR_URL ?= http://localhost:8080/solr/ 24,27c24,27 < OWLTOOLS_ROOT ?= /nfs0/BPP/Jaiswal_Lab/justin/owltools/ < BBOP_JS_ROOT ?= /data/www/planteome_dev/amigo/external/ < #PANTHER_FILES_DIR ?= ~/local/src/svn/geneontology.org/trunk/experimental/trees/panther_data/ < #SOLR_DATA_ROOT ?= /srv/solr --- > OWLTOOLS_ROOT ?= ~/local/src/git/owltools/ > BBOP_JS_ROOT ?= ~/local/src/git/bbop-js/ > PANTHER_FILES_DIR ?= ~/local/src/svn/geneontology.org/trunk/experimental/trees/panther_data/ > SOLR_DATA_ROOT ?= /srv/solr
load-ontology
Try to run the load-ontology part of the Makefile:
make load-ontology
This will take some time as it appears to be updating the owltools and then loading the ontologies
At this point, I get an error when trying to load the ontologies. Working with Seth to figure it out.
Errors were mostly caused by memory issues. Had to comment out the ncbitaxon.owl ontology from the list of ones ran. Upped the memory to 20GB on palea. May eventually have to look at running the load on one of the cluster nodes as they have much more RAM. Would have to figure out how to open up the SOLR port to the internal network though.
Now to get annotation test data into GOLR
make load-test-gaf
This took a few minutes to run, not long.
At this point, data for an initial test is loaded into SOLR, next is to get AmiGO2 up and running off that instance.
AmiGO2 install
Since we already have pulled the amigo code from git, we have to work on configuring it:
cd amigo/conf cp .initial_values.yaml amigo.yaml
Change the amigo.yaml file as follows:
[elserj@palea conf]$ diff amigo.yaml .initial_values.yaml
5c5
< value: http://planteome.org/amigo/cgi-bin/crop-amigo
---
> value: http://amigo1.geneontology.org/cgi-bin/amigo
17c17
< value: /data/www/planteome_dev/amigo/perl/bin
---
> value: /home/sjcarbon/local/src/git/amigo/perl/bin
33c33
< value: http://localhost:8983/solr/
---
> value: http://localhost:8080/solr/
37c37
< value: /data/www/planteome_dev/amigo
---
> value: /home/sjcarbon/local/src/git/amigo
41c41
< value: planteome-dev
---
> value: localhost
45c45
< value: /data/www/planteome_dev/amigo/static
---
> value: /home/sjcarbon/local/src/git/amigo/static
53c53
< value: 'all|[GPTE]O:[u0-9]{7}'
---
> value: 'all|GO:[0-9]{7}'
65c65
< value: /data/www/planteome_dev/amigo/golr_timestamp.log
---
> value: /home/sjcarbon/local/src/git/amigo/golr_timestamp.log
69c69
< value: /data/planteome_dev/amigo/external/bbop.js
---
> value: /home/sjcarbon/local/src/git/amigo/external/bbop.js
73c73
< value: /data/www/planteome_dev/amigo/metadata
---
> value: /home/sjcarbon/local/src/git/amigo/metadata
Now to run
make install
This gives an error about a missing yui-compressor, so need to get that installed.
sudo mkdir /usr/local/share/yui-compressor sudo chown elserj: /usr/local/share/yui-compressor cd /usr/local/share/yui-compressor wget https://github.com/downloads/yui/yuicompressor/yuicompressor-2.4.7.zip unzip yuicompressor-2.4.7.zip
Create the following script in /usr/local/bin/yui-comprssor
#!/bin/sh # Quick and dirty front-end script to the yui-compressor that emulates # /usr/bin/yuicompressor script from Ubuntu release # JAVA_HOME=/usr/java/default YUI_JAR=/usr/local/share/yui-compressor/yuicompressor-2.4.7/build/yuicompressor-2.4.7.jar # Run YUI Compressor /usr/bin/java -jar $YUI_JAR "$@"
Make it executable with
sudo chmod +x /usr/local/bin/yui-compressor
Next problem is that rhino gives an error message about "invalid option modules". Still communicating with Seth on solution. Will update when solution found.
Solution was to comment out the test and docs part of the install target in the Makefile. Rhino was apparently only used for unit testing, which we don't really need to do.
Configure Apache2
The Apache config file needs to be modified
[elserj@palea amigo]$ diff conf/amigo2-apache.conf conf/amigo2-apache.conf.bla 2,3c2,3 < <VirtualHost *:9999> < ServerAdmin sjcarbon@localhost --- > <VirtualHost *:80> > ServerAdmin elserj@localhost > ServerName dev.planteome.org 7c7 < DocumentRoot /home/sjcarbon/local/src/git/amigo --- > DocumentRoot /data/www/planteome_dev/amigo 10c10 < <Directory /home/sjcarbon/local/src/git/amigo/static> --- > <Directory /data/www/planteome_dev/amigo/static> 19c19 < ScriptAlias /$binnm /home/sjcarbon/local/src/git/amigo/perl/bin/$binnm --- > ScriptAlias /$binnm /data/www/planteome_dev/amigo/perl/bin/$binnm 47,48c47,48 < ErrorLog /var/log/apache2/error.log < ScriptLog /var/log/apache2/cgi.log --- > ErrorLog /data/www/planteome_dev/amigo/log/error.log > ScriptLog /data/www/planteome_dev/amigo/log/cgi.log 54c54 < CustomLog /var/log/apache2/access.log combined --- > CustomLog /data/www/planteome_dev/amigo/log/access.log combined
For this to work, the mod_macro module for Apache must be installed. I found it as a downloadable rpm, so I installed it that way:
ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel6/x86_64/mod_macro-1.1.11-2.el6.pp.x86_64.rpm sudo rpm -Uvih mod_macro-1.1.11-2.el6.pp.x86_64.rpm
I changed the /etc/httpd/conf/httpd.conf file to have the following line for the dev.planteome.org section
Include /data/www/planteome_dev/amigo/amigo2-apache-local.conf
Note that this cannot go in it's own VirtualHost section as it will error because of a VirtualHost inside a VirtualHost
doing the
make install
now works without error.
However, the server doesn't work when navigated to. It looks like a perl module is missing. Installed the following via cpan:
CGI::Application::Plugin::Forward
Error is now about "Type of arg 1 to keys must be hash" Will work on this error.
This error apparently is the result of the older version of perl that we have installed. Seth fixed this in git, so I did a git pull and got past it.
However, another perl module needed to be installed via cpan:
CGI::Application::Dispatch
At this point Amigo2 appears to be working.
This page will be updated as I progress.