AmiGO2 install: Difference between revisions

From Planteome.org
Jump to navigation Jump to search
(Created page with "== GOLR install: == There are a few prerequisites to getting GOLR installed. They appear to be Maven and OwlTools. To install maven, I followed the instructions from here: h...")
 
 
(28 intermediate revisions by the same user not shown)
Line 1: Line 1:
== GOLR install: ==
== GOLR install: ==
There are a few prerequisites to getting GOLR installed.  They appear to be Maven and OwlTools.  To install maven, I followed the instructions from here: http://blog.gluster.org/2013/08/yum-install-maven-yes-you-can/, seemed to work without error.
Install the 3.6 version of SOLR


Then, to install OWLTools, I did the following:
===SOLR===
cd /lemma/justin
Grab the 3.6 version of SOLR from apache and unpack:
svn co http://owltools.googlecode.com/svn/trunk/ owltools
<pre>
This creates the owltools directory in my space on lemma.
wget http://archive.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
tar zxvf apache-solr-3.6.2.tgz
</pre>


To compile OWLTools, run:
Pull down the latest version of AmiGO:
mvn clean package
<pre>
cd /data/www/planteome_dev
git clone https://github.com/planteome/amigo.git amigo
</pre>


Go back to the SOLR directory and replace the config with those from GOLR:
<pre>
cd /data/www/planteome_dev/SOLR/example/solr/conf
cp schema.xml schema.xml.org
cp solrconfig.xml solrconfig.xml.org
cp /data/www/planteome_dev/amigo/golr/solr/conf/s* ./
</pre>
I don't see any edits that need to be made at this point.  I think when we move to additional ontologies, we will have to modify schema.xml to account for new relationship types.  Cross that bridge when we get there.
Start SOLR(GOLR):
<pre>
cd ../../ (be in the example directory)
java -jar start.jar
</pre>
Now that it seems to be starting up fine, daemonise it so that it runs at a service at boot (Note that any service needs to be okayed by Chris before started).  Pick either the systemd or init.d version, depending on which init system is being used:
systemd version:
<pre>
[elserj@floret extensions]$ cat /etc/systemd/system/solr.service
[Unit]
Description=Apache Solr
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
Type=simple
WorkingDirectory=/data/www/planteome/SOLR/example
ExecStart=/usr/bin/java -jar /data/www/planteome/SOLR/example/start.jar
Restart=on-failure
[Install]
WantedBy=multi-user.target
</pre>
init.d version:
Add the init.d script as outlined [http://stackoverflow.com/questions/2150767/how-to-start-solr-automatically on this page]
Had to modify that script as follows to make it work on CentOS:
<pre>
[elserj@palea amigo]$ diff /etc/init.d/solr /etc/init.d/solr.org
16,19d15
< # Source function library.
< [ -f /etc/rc.d/init.d/functions ] || exit 0
< . /etc/rc.d/init.d/functions
<
24c20
<    daemon "cd /data/www/planteome_dev/SOLR/example; java -jar /data/www/planteome_dev/SOLR/example/start.jar &> /var/log/solr/solr.log &"
---
>    daemon --chdir='/data/www/plantome_dev/example' --command "java -jar start.jar" --respawn --output=/var/log/solr/solr.log --name=solr --verbose
40c36
<    ps -aef | grep -v planteome_dev2 | grep planteome_dev | grep -v grep | awk '{print $2}' | xargs -i++ kill -9 ++
---
>    daemon --stop --name=solr  --verbose
</pre>
At this point, SOLR is now running.  Next is to load data to it.
==AmiGO2 install==
Copy the example amigo.yaml file and make the following changes:
<pre>
[elserj@palea conf]$ diff amigo.yaml examples/amigo.yaml.localhost-planteome
5c5
<  value: http://planteome.org/amigo/cgi-bin/crop-amigo/go.cgi
---
>  value: http://amigo1.geneontology.org/cgi-bin/amigo
17c17
<  value: /data/www/planteome_dev2/amigo/perl/bin
---
>  value: /home/sjcarbon/local/src/git/amigo/perl/bin
21c21
<  value: http://test.planteome.org
---
>  value: http://localhost:9999
32a33,37
>    - model_annotation
>    - noctua_model_meta
>    - family
>    - general
>    - bbop_term_ac
40c45
<  value: http://galaxy.planteome.org/
---
>  value: http://galaxy.berkeleybop.org/
42c47
<  comment: 'The public URL of the GOlr server (including slash); this is the one called by AmiGO resource, JS, etc.'
---
>  comment: The public URL of the GOlr server (including slash); this is the one called by AmiGO resource, JS, etc.
44c49
<  value: http://test.planteome.org/solr/
---
>  value: http://dev.planteome.org/solr/
48c53
<  value: http://noctua.planteome.org/
---
>  value: http://noctua.berkeleybop.org/
52c57
<  value: /data/www/planteome_dev2/amigo
---
>  value: /home/sjcarbon/local/src/git/amigo
56c61
<  value: test.planteome.org
---
>  value: localhost
60c65
<  value: /data/www/planteome_dev2/amigo/static
---
>  value: /home/sjcarbon/local/src/git/amigo/static
64c69
<  value: http://test.planteome.org/static
---
>  value: http://localhost:9999/static
68c73
<  value: 'all|[PGTE]O:[0-9]{7}'
---
>  value: 'all|GO:[0-9]{7}'
76c81
<  value: 1.0
---
>  value: 2.3.2
78c83
<  comment: 'Please enter the full path to readable/writable directory that will be used for things like temporary files, logs (if enabled), and the golr_timestamp.log file.'
---
>  comment: Please enter the full path to readable/writable directory that will be used for things like temporary files, logs (if enabled), and the golr_timestamp.log file.
89,91c94,98
<    - file:///data/www/planteome_dev/amigo/associations/eo-associations/eo_diversity_arabidopsis.assoc
<    - file:///data/www/planteome_dev/amigo/associations/po-associations/po_anatomy_gene_arabidopsis_tair.assoc
<    - file:///data/www/planteome_dev/amigo/associations/to-associations/to_diversity_arabidopsis.assoc
---
>    - http://www.geneontology.org/gene-associations/gene_association.mgi.gz
>    - http://www.geneontology.org/gene-associations/gene_association.goa_human.gz
>    - http://www.geneontology.org/gene-associations/gene_association.goa_ref_human.gz
>    - http://www.geneontology.org/gene-associations/gene_association.goa_uniprot_noiea.gz
>    - http://www.geneontology.org/gene-associations/gene_association.reactome.gz
96,99c103,110
<    - /data/www/planteome_dev/amigo/metadata/ont-config.yaml
<    - /data/www/planteome_dev/amigo/metadata/bio-config.yaml
<    - /data/www/planteome_dev/amigo/metadata/ann-config.yaml
<    - /data/www/planteome_dev/amigo/metadata/general-config.yaml
---
>    - '~/local/src/git/amigo/metadata/ont-config.yaml'
>    - '~/local/src/git/amigo/metadata/bio-config.yaml'
>    - '~/local/src/git/amigo/metadata/ann-config.yaml'
>    - '~/local/src/git/amigo/metadata/general-config.yaml'
>    - '~/local/src/git/amigo/metadata/model-ann-config.yaml'
>    - '~/local/src/git/amigo/metadata/noctua-model-meta-config.yaml'
>    - '~/local/src/git/amigo/metadata/protein-family-config.yaml'
>    - '~/local/src/git/amigo/metadata/ann_ev_agg-config.yaml'
103c114
<  value: /data/www/planteome_dev2/amigo/metadata/ont-config.yaml
---
>  value: '~/local/src/git/amigo/metadata/ont-config.yaml'
107c118
<  value: /data/www/planteome_dev2/noctua-models/models/
---
>  value: '~/local/src/git/noctua-models/models/'
112c123
<    - file:///data/www/planteome_dev2/amigo/ontology_files/planteome.owl
---
>    - http://purl.obolibrary.org/obo/go/extensions/go-plus.owl
120c131
<  value: 20G
---
>  value: 3G
</pre>
Next is to install nodejs (if it isn't already)
On CentOS 6:
<pre>sudo su -
curl --silent --location https://rpm.nodesource.com/setup | bash -
yum install nodejs</pre>
Now to run <pre>npm install</pre> to install necessary nodejs modules.
===Configure Apache2===
The Apache config file needs to be modified
<pre>
[elserj@palea amigo]$ diff amigo2-apache-local.conf conf/amigo2-apache.conf
2,4c2,3
< <VirtualHost *:80>
<    ServerAdmin elserj@science.oregonstate.edu
<    ServerName dev.planteome.org
---
> <VirtualHost *:9999>
>    ServerAdmin sjcarbon@localhost
8c7
<    DocumentRoot /data/www/planteome_dev/amigo
---
>    DocumentRoot /home/sjcarbon/local/src/git/amigo
11c10
<    <Directory /data/www/planteome_dev/amigo/static>
---
>    <Directory /home/sjcarbon/local/src/git/amigo/static>
20c19
<      ScriptAlias /$binnm /data/www/planteome_dev/amigo/perl/bin/$binnm
---
>      ScriptAlias /$binnm /home/sjcarbon/local/src/git/amigo/perl/bin/$binnm
48,59c47,48
<
<  ProxyRequests Off
<        <Proxy *>
<        Order deny,allow
<        Allow from 192.168.1.52
<        </Proxy>
<
<    ProxyPass /solr http://localhost:8983/solr
<    ProxyPassReverse /solr http://localhost:8983/solr
<
<    ErrorLog /data/www/planteome_dev/amigo/log/error.log
<    ScriptLog /data/www/planteome_dev/amigo/log/cgi.log
---
>    ErrorLog /var/log/apache2/error.log
>    ScriptLog /var/log/apache2/cgi.log
65c54
<    CustomLog /data/www/planteome_dev/amigo/log/access.log combined
---
>    CustomLog /var/log/apache2/access.log combined</pre>
For this to work, the mod_macro module for Apache must be installed.  I found it as a downloadable rpm, so I installed it that way:
<pre>wget ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel6/x86_64/mod_macro-1.1.11-2.el6.pp.x86_64.rpm
sudo rpm -Uvih mod_macro-1.1.11-2.el6.pp.x86_64.rpm</pre>
The log files need to be created or else Apache will not start:
<pre>mkdir log
touch log/access.log
touch log/error.log
sudo chown root: log/*.log
</pre>
The proxy settings are to make it so the Server is able to talk to the SOLR data.
I changed the /etc/httpd/conf/httpd.conf file to have the following line for the dev.planteome.org section
<pre>Include /data/www/planteome_dev/amigo/amigo2-apache-local.conf</pre>
Note that this cannot go in it's own VirtualHost section as it will error because of a VirtualHost inside a VirtualHost
===Run amigo2 installer===
<pre>./node_modules/.bin/gulp install</pre>
There may be some missing perl packages here.  I had to install <pre>Config::YAML
Template
Date::Parse
Clone
URI::Escape
JSON::XS
Data::UUID
File::Type
FreezeThaw
WWW::Mechanize
LWP::Protocol::https
CGI::Application::Dispatch
CGI::Application::Plugin::Forward
CGI::Application::Plugin::Session
CGI::Application::Plugin::TT
CGI::Application::Plugin::Redirect
DBI
Data::FormValidator
Graph::Directed
XML::Parser
XML::XPath
GraphViz
DBD::SQLite
</pre>
===Load ontologies===
<pre>./node_modules/.bin/gulp load-ontology</pre>
===Load associations===
<pre>./node_modules/.bin/gulp load-gafs</pre>
At this point Amigo2 appears to be working.
==Notes from older install==
At one point from doing an older install, it needed a missing yui-compressor, so need to get that installed. (May no longer be relevant)
<pre>
sudo mkdir /usr/local/share/yui-compressor
sudo chown elserj: /usr/local/share/yui-compressor
cd /usr/local/share/yui-compressor
wget https://github.com/downloads/yui/yuicompressor/yuicompressor-2.4.7.zip
unzip yuicompressor-2.4.7.zip
</pre>
Create the following script in /usr/local/bin/yui-comprssor
<pre>
#!/bin/sh
# Quick and dirty front-end script to the yui-compressor that emulates
# /usr/bin/yuicompressor script from Ubuntu release
#
JAVA_HOME=/usr/java/default
YUI_JAR=/usr/local/share/yui-compressor/yuicompressor-2.4.7/build/yuicompressor-2.4.7.jar
# Run YUI Compressor
/usr/bin/java -jar $YUI_JAR "$@"
</pre>
Make it executable with <pre>sudo chmod +x /usr/local/bin/yui-compressor</pre>




'''This page will be updated as I progress.'''
'''This page will be updated as I progress.'''

Latest revision as of 18:46, 2 November 2016

GOLR install:

Install the 3.6 version of SOLR

SOLR

Grab the 3.6 version of SOLR from apache and unpack: 

wget http://archive.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
tar zxvf apache-solr-3.6.2.tgz

Pull down the latest version of AmiGO: 

cd /data/www/planteome_dev
git clone https://github.com/planteome/amigo.git amigo

Go back to the SOLR directory and replace the config with those from GOLR: 

cd /data/www/planteome_dev/SOLR/example/solr/conf
cp schema.xml schema.xml.org
cp solrconfig.xml solrconfig.xml.org
cp /data/www/planteome_dev/amigo/golr/solr/conf/s* ./

I don't see any edits that need to be made at this point. I think when we move to additional ontologies, we will have to modify schema.xml to account for new relationship types. Cross that bridge when we get there.


Start SOLR(GOLR): 

cd ../../ (be in the example directory)
java -jar start.jar

Now that it seems to be starting up fine, daemonise it so that it runs at a service at boot (Note that any service needs to be okayed by Chris before started). Pick either the systemd or init.d version, depending on which init system is being used:

systemd version: 

[elserj@floret extensions]$ cat /etc/systemd/system/solr.service 
[Unit]
Description=Apache Solr
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
WorkingDirectory=/data/www/planteome/SOLR/example
ExecStart=/usr/bin/java -jar /data/www/planteome/SOLR/example/start.jar
Restart=on-failure

[Install]
WantedBy=multi-user.target

init.d version: Add the init.d script as outlined on this page

Had to modify that script as follows to make it work on CentOS: 

[elserj@palea amigo]$ diff /etc/init.d/solr /etc/init.d/solr.org 
16,19d15
< # Source function library.
< [ -f /etc/rc.d/init.d/functions ] || exit 0
< . /etc/rc.d/init.d/functions
< 
24c20
<     daemon "cd /data/www/planteome_dev/SOLR/example; java -jar /data/www/planteome_dev/SOLR/example/start.jar &> /var/log/solr/solr.log &"
---
>     daemon --chdir='/data/www/plantome_dev/example' --command "java -jar start.jar" --respawn --output=/var/log/solr/solr.log --name=solr --verbose
40c36
<     ps -aef | grep -v planteome_dev2 | grep planteome_dev | grep -v grep | awk '{print $2}' | xargs -i++ kill -9 ++
---
>     daemon --stop --name=solr  --verbose

At this point, SOLR is now running. Next is to load data to it.

AmiGO2 install

Copy the example amigo.yaml file and make the following changes: 

 
[elserj@palea conf]$ diff amigo.yaml examples/amigo.yaml.localhost-planteome 
5c5
<   value: http://planteome.org/amigo/cgi-bin/crop-amigo/go.cgi
---
>   value: http://amigo1.geneontology.org/cgi-bin/amigo
17c17
<   value: /data/www/planteome_dev2/amigo/perl/bin
---
>   value: /home/sjcarbon/local/src/git/amigo/perl/bin
21c21
<   value: http://test.planteome.org
---
>   value: http://localhost:9999
32a33,37
>     - model_annotation
>     - noctua_model_meta
>     - family
>     - general
>     - bbop_term_ac
40c45
<   value: http://galaxy.planteome.org/
---
>   value: http://galaxy.berkeleybop.org/
42c47
<   comment: 'The public URL of the GOlr server (including slash); this is the one called by AmiGO resource, JS, etc.'
---
>   comment: The public URL of the GOlr server (including slash); this is the one called by AmiGO resource, JS, etc.
44c49
<   value: http://test.planteome.org/solr/
---
>   value: http://dev.planteome.org/solr/
48c53
<   value: http://noctua.planteome.org/
---
>   value: http://noctua.berkeleybop.org/
52c57
<   value: /data/www/planteome_dev2/amigo
---
>   value: /home/sjcarbon/local/src/git/amigo
56c61
<   value: test.planteome.org
---
>   value: localhost
60c65
<   value: /data/www/planteome_dev2/amigo/static
---
>   value: /home/sjcarbon/local/src/git/amigo/static
64c69
<   value: http://test.planteome.org/static
---
>   value: http://localhost:9999/static
68c73
<   value: 'all|[PGTE]O:[0-9]{7}'
---
>   value: 'all|GO:[0-9]{7}'
76c81
<   value: 1.0
---
>   value: 2.3.2
78c83
<   comment: 'Please enter the full path to readable/writable directory that will be used for things like temporary files, logs (if enabled), and the golr_timestamp.log file.'
---
>   comment: Please enter the full path to readable/writable directory that will be used for things like temporary files, logs (if enabled), and the golr_timestamp.log file.
89,91c94,98
<     - file:///data/www/planteome_dev/amigo/associations/eo-associations/eo_diversity_arabidopsis.assoc
<     - file:///data/www/planteome_dev/amigo/associations/po-associations/po_anatomy_gene_arabidopsis_tair.assoc
<     - file:///data/www/planteome_dev/amigo/associations/to-associations/to_diversity_arabidopsis.assoc
---
>     - http://www.geneontology.org/gene-associations/gene_association.mgi.gz
>     - http://www.geneontology.org/gene-associations/gene_association.goa_human.gz
>     - http://www.geneontology.org/gene-associations/gene_association.goa_ref_human.gz
>     - http://www.geneontology.org/gene-associations/gene_association.goa_uniprot_noiea.gz
>     - http://www.geneontology.org/gene-associations/gene_association.reactome.gz
96,99c103,110
<     - /data/www/planteome_dev/amigo/metadata/ont-config.yaml
<     - /data/www/planteome_dev/amigo/metadata/bio-config.yaml
<     - /data/www/planteome_dev/amigo/metadata/ann-config.yaml
<     - /data/www/planteome_dev/amigo/metadata/general-config.yaml
---
>     - '~/local/src/git/amigo/metadata/ont-config.yaml'
>     - '~/local/src/git/amigo/metadata/bio-config.yaml'
>     - '~/local/src/git/amigo/metadata/ann-config.yaml'
>     - '~/local/src/git/amigo/metadata/general-config.yaml'
>     - '~/local/src/git/amigo/metadata/model-ann-config.yaml'
>     - '~/local/src/git/amigo/metadata/noctua-model-meta-config.yaml'
>     - '~/local/src/git/amigo/metadata/protein-family-config.yaml'
>     - '~/local/src/git/amigo/metadata/ann_ev_agg-config.yaml'
103c114
<   value: /data/www/planteome_dev2/amigo/metadata/ont-config.yaml
---
>   value: '~/local/src/git/amigo/metadata/ont-config.yaml'
107c118
<   value: /data/www/planteome_dev2/noctua-models/models/
---
>   value: '~/local/src/git/noctua-models/models/'
112c123
<     - file:///data/www/planteome_dev2/amigo/ontology_files/planteome.owl
---
>     - http://purl.obolibrary.org/obo/go/extensions/go-plus.owl
120c131
<   value: 20G
---
>   value: 3G

Next is to install nodejs (if it isn't already)

On CentOS 6: 

sudo su -
curl --silent --location https://rpm.nodesource.com/setup | bash -
yum install nodejs

Now to run 

npm install

to install necessary nodejs modules.


Configure Apache2

The Apache config file needs to be modified 

[elserj@palea amigo]$ diff amigo2-apache-local.conf conf/amigo2-apache.conf
2,4c2,3
< <VirtualHost *:80>
<     ServerAdmin elserj@science.oregonstate.edu
<     ServerName dev.planteome.org
---
> <VirtualHost *:9999>
>     ServerAdmin sjcarbon@localhost
8c7
<     DocumentRoot /data/www/planteome_dev/amigo
---
>     DocumentRoot /home/sjcarbon/local/src/git/amigo
11c10
<     <Directory /data/www/planteome_dev/amigo/static>
---
>     <Directory /home/sjcarbon/local/src/git/amigo/static>
20c19
<       ScriptAlias /$binnm /data/www/planteome_dev/amigo/perl/bin/$binnm
---
>       ScriptAlias /$binnm /home/sjcarbon/local/src/git/amigo/perl/bin/$binnm
48,59c47,48
< 
<  ProxyRequests Off
<         <Proxy *>
<         Order deny,allow
<         Allow from 192.168.1.52
<         </Proxy>
< 
<     ProxyPass /solr http://localhost:8983/solr
<     ProxyPassReverse /solr http://localhost:8983/solr
< 
<     ErrorLog /data/www/planteome_dev/amigo/log/error.log
<     ScriptLog /data/www/planteome_dev/amigo/log/cgi.log
---
>     ErrorLog /var/log/apache2/error.log
>     ScriptLog /var/log/apache2/cgi.log
65c54
<     CustomLog /data/www/planteome_dev/amigo/log/access.log combined
---
>     CustomLog /var/log/apache2/access.log combined

For this to work, the mod_macro module for Apache must be installed. I found it as a downloadable rpm, so I installed it that way: 

wget ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel6/x86_64/mod_macro-1.1.11-2.el6.pp.x86_64.rpm
sudo rpm -Uvih mod_macro-1.1.11-2.el6.pp.x86_64.rpm


The log files need to be created or else Apache will not start: 

mkdir log
touch log/access.log
touch log/error.log
sudo chown root: log/*.log

The proxy settings are to make it so the Server is able to talk to the SOLR data.

I changed the /etc/httpd/conf/httpd.conf file to have the following line for the dev.planteome.org section 

Include /data/www/planteome_dev/amigo/amigo2-apache-local.conf

Note that this cannot go in it's own VirtualHost section as it will error because of a VirtualHost inside a VirtualHost

Run amigo2 installer

./node_modules/.bin/gulp install

There may be some missing perl packages here. I had to install 

Config::YAML
Template
Date::Parse
Clone
URI::Escape
JSON::XS
Data::UUID
File::Type
FreezeThaw
WWW::Mechanize
LWP::Protocol::https
CGI::Application::Dispatch
CGI::Application::Plugin::Forward
CGI::Application::Plugin::Session
CGI::Application::Plugin::TT
CGI::Application::Plugin::Redirect
DBI
Data::FormValidator
Graph::Directed
XML::Parser
XML::XPath
GraphViz
DBD::SQLite

Load ontologies

./node_modules/.bin/gulp load-ontology

Load associations

./node_modules/.bin/gulp load-gafs


At this point Amigo2 appears to be working.

Notes from older install

At one point from doing an older install, it needed a missing yui-compressor, so need to get that installed. (May no longer be relevant) 

sudo mkdir /usr/local/share/yui-compressor
sudo chown elserj: /usr/local/share/yui-compressor
cd /usr/local/share/yui-compressor
wget https://github.com/downloads/yui/yuicompressor/yuicompressor-2.4.7.zip
unzip yuicompressor-2.4.7.zip

Create the following script in /usr/local/bin/yui-comprssor 

#!/bin/sh
# Quick and dirty front-end script to the yui-compressor that emulates
# /usr/bin/yuicompressor script from Ubuntu release
#
JAVA_HOME=/usr/java/default
YUI_JAR=/usr/local/share/yui-compressor/yuicompressor-2.4.7/build/yuicompressor-2.4.7.jar
# Run YUI Compressor
/usr/bin/java -jar $YUI_JAR "$@"

Make it executable with 

sudo chmod +x /usr/local/bin/yui-compressor


This page will be updated as I progress.

Retrieved from "https://wiki.planteome.org/index.php?title=AmiGO2_install&oldid=3099"