Svn migration to git - Revision history

Elserj: /* Filtering the dump */

2015-08-22T04:18:59Z

Filtering the dump

← Older revision		Revision as of 04:18, 22 August 2015
Line 33:		Line 33:
	<pre>		<pre>
	cat trait.obo_filtered_dump \| sed 's/Node-path: trunk\/ontology\/collaborators_ontology\/gramene\/traits\//Node-path: plant-trait-ontology.obo/' > trait.obo_filtered_path_fixed_dump		cat trait.obo_filtered_dump \| sed 's/Node-path: trunk\/ontology\/collaborators_ontology\/gramene\/traits\//Node-path: plant-trait-ontology.obo/' > trait.obo_filtered_path_fixed_dump
			</pre>

			For repos that have had files moved with history before, the above did not work when trying to import into repo in the next step. I used the following for a different repo to get it to work:
			<pre>cat associations_dump \| sed 's/Node-path: trunk\/associations/Node-path: po-associations/' \| sed 's/Node-copyfrom-path: trunk\/associations/Node-copyfrom-path: po-associations/' > associations_filtered_path_dump
	</pre>		</pre>

Elserj: /* Filtering the dump */

2015-08-22T00:37:32Z

Filtering the dump

← Older revision		Revision as of 00:37, 22 August 2015
Line 32:		Line 32:
	If the file path needs to be changed, this is the point to do it. Here is a command that I used to change the filename and remove the full path:		If the file path needs to be changed, this is the point to do it. Here is a command that I used to change the filename and remove the full path:
	<pre>		<pre>
	cat trait.obo_filtered_dump \| sed 's/trunk\/ontology\/collaborators_ontology\/gramene\/traits\//plant-trait-ontology.obo/' > trait.obo_filtered_path_fixed_dump		cat trait.obo_filtered_dump \| sed 's/Node-path: trunk\/ontology\/collaborators_ontology\/gramene\/traits\//Node-path: plant-trait-ontology.obo/' > trait.obo_filtered_path_fixed_dump
	</pre>		</pre>

Elserj: /* Method if github repo already has files in it */

2015-08-18T03:12:46Z

Method if github repo already has files in it

← Older revision		Revision as of 03:12, 18 August 2015
Line 71:		Line 71:
	cd common-files-for-ref-ontologies/		cd common-files-for-ref-ontologies/
	git remote add origin-dbxref ../dbxref		git remote add origin-dbxref ../dbxref
	~~git pull origin-dbxref~~
	git pull origin-dbxref master		git pull origin-dbxref master
	git remote rm origin-dbxref		git remote rm origin-dbxref

Elserj: /* Copy the svn repo to github */

2015-07-30T22:46:03Z

Copy the svn repo to github

← Older revision		Revision as of 22:46, 30 July 2015
Line 68:		Line 68:
	git commit		git commit
	cd ../		cd ../
	~~git clone https://github.com/Planteome/common-files-ref-ontologies.git common-files-ref-ontologies~~
	git clone https://github.com/Planteome/common-files-for-ref-ontologies.git common-files-for-ref-ontologies		git clone https://github.com/Planteome/common-files-for-ref-ontologies.git common-files-for-ref-ontologies
	cd common-files-for-ref-ontologies/		cd common-files-for-ref-ontologies/

Elserj at 22:45, 30 July 2015

2015-07-30T22:45:30Z

← Older revision		Revision as of 22:45, 30 July 2015
Line 51:		Line 51:

	==Copy the svn repo to github==		==Copy the svn repo to github==
	~~Checkout~~ the repo from github		===Method if repo in github hasn't been created yet (simpler)===
			Go to https://import.github.com/new and follow the steps there. This will use the SVN repo created above.

			===Method if github repo already has files in it===
			I couldn't figure out how to do this direct from the svn, but I figured out how to do it by using an intermediate temporary github repo.

			Create a new temporary github repo from the svn using the "simpler" import method described above.

			In this example, I created a temporary github repo https://github.com/elser/dbxref that contained just the PO_DBXref.txt file along with its history. I wanted to move it to the Planteome/common-files-for-ref-ontologies repo under Planteome.
			<pre>mkdir git_temp
			cd git_temp/
			git clone https://github.com/elserj/dbxref.git dbxref
			cd dbxref/
			git remote rm origin
			git filter-branch --subdirectory-filter dbxref -- --all
			git commit
			cd ../
			git clone https://github.com/Planteome/common-files-ref-ontologies.git common-files-ref-ontologies
			git clone https://github.com/Planteome/common-files-for-ref-ontologies.git common-files-for-ref-ontologies
			cd common-files-for-ref-ontologies/
			git remote add origin-dbxref ../dbxref
			git pull origin-dbxref
			git pull origin-dbxref master
			git remote rm origin-dbxref
			git push</pre>

			Note that not every line is required, noticeably the "git filter-branch", but they may be needed if the repo has multiple folders or other structure that this example didn't have.

Elserj: /* Copy the filtered dump to the new repo */

2015-07-30T18:05:33Z

Copy the filtered dump to the new repo

← Older revision		Revision as of 18:05, 30 July 2015
Line 41:		Line 41:

	==Copy the filtered dump to the new repo==		==Copy the filtered dump to the new repo==
	~~Before we can add the revisions to the repo, we need to at least create the folder where the fitered dump files are in the new repo, or we will get an error:~~
	~~<pre>cd ~/palea_svn~~
	~~svn co http://palea.cgrb.oregonstate.edu:/svn/svn2git~~
	~~cd svn2git~~
	~~mkdir -p trunk/ontology/collaborators_ontology/gramene/traits~~
	~~svn add trunk~~
	~~svn commit --username=elserj~~
	~~</pre>~~
	Then back on palea (svn server):		Then back on palea (svn server):
	<pre>cd /data/svnrepos		<pre>cd /data/svnrepos
	sudo svnadmin load ./svn2git < /lemma/justin/Poc_svn_migration/trait.~~obo_filtered_dump~~		sudo svnadmin load ./svn2git < /lemma/justin/Poc_svn_migration/trait.obo_filtered_path_fixed_dump
	</pre>		</pre>

	~~Fix the file path (from the svn client):~~
	~~<pre>svn up~~
	~~svn mv trunk/ontology/collaborators_ontology/gramene/traits/trait.obo ./~~
	~~svn up~~
	~~svn rm trunk~~
	~~svn commit</pre>~~

Elserj: /* Filtering the dump */

2015-07-30T00:10:08Z

Filtering the dump

← Older revision		Revision as of 00:10, 30 July 2015
Line 29:		Line 29:

	The <pre>svndumpfilter</pre> command will go through the dump file and only output revision info for trait.obo, in this case. The full path needs to be in the include argument, otherwise it output an empty repo. The other arguments will reset the revision numbers so that they start at 1 and have no empty revisions.		The <pre>svndumpfilter</pre> command will go through the dump file and only output revision info for trait.obo, in this case. The full path needs to be in the include argument, otherwise it output an empty repo. The other arguments will reset the revision numbers so that they start at 1 and have no empty revisions.

			If the file path needs to be changed, this is the point to do it. Here is a command that I used to change the filename and remove the full path:
			<pre>
			cat trait.obo_filtered_dump \| sed 's/trunk\/ontology\/collaborators_ontology\/gramene\/traits\//plant-trait-ontology.obo/' > trait.obo_filtered_path_fixed_dump
			</pre>

	==Create a new svn repo to hold the filtered data==		==Create a new svn repo to hold the filtered data==

Elserj: /* Filtering the dump */

2015-07-30T00:03:20Z

Filtering the dump

← Older revision		Revision as of 00:03, 30 July 2015
Line 25:		Line 25:
	I decided to test this out with trait.obo first as a proof of concept. Other files should be similar with changes to the include statement		I decided to test this out with trait.obo first as a proof of concept. Other files should be similar with changes to the include statement
	<pre>cd /lemma/justin/Poc_svn_migration		<pre>cd /lemma/justin/Poc_svn_migration
	cat Poc_svn_dump \| svndumpfilter include trunk/ontology/collaborators_ontology/gramene/traits/trait.obo --drop-empty-revs --renumber-revs > trait.obo_filtered_dump		cat Poc_svn_dump \| svndumpfilter include /trunk/ontology/collaborators_ontology/gramene/traits/trait.obo --drop-empty-revs --renumber-revs > trait.obo_filtered_dump
	</pre>		</pre>

Elserj: /* Dumping the svn repo */

2015-07-29T23:39:46Z

Dumping the svn repo

← Older revision		Revision as of 23:39, 29 July 2015
Line 15:		Line 15:
	On Palea		On Palea
	<pre>cd /data/svnrepos/Poc		<pre>cd /data/svnrepos/Poc
	~~svn admin~~ dump . > /lemma/justin/Poc_svn_migration/Poc_svn_dump		svnadmin dump . > /lemma/justin/Poc_svn_migration/Poc_svn_dump
	</pre>		</pre>

Elserj at 18:53, 1 May 2015

2015-05-01T18:53:25Z

← Older revision		Revision as of 18:53, 1 May 2015
Line 29:		Line 29:

	The <pre>svndumpfilter</pre> command will go through the dump file and only output revision info for trait.obo, in this case. The full path needs to be in the include argument, otherwise it output an empty repo. The other arguments will reset the revision numbers so that they start at 1 and have no empty revisions.		The <pre>svndumpfilter</pre> command will go through the dump file and only output revision info for trait.obo, in this case. The full path needs to be in the include argument, otherwise it output an empty repo. The other arguments will reset the revision numbers so that they start at 1 and have no empty revisions.

			==Create a new svn repo to hold the filtered data==
			<pre>cd /data/svnrepos
			sudo svnadmin create /data/svnrepos/svn2git
			sudo chown -R apache: svn2git</pre>

			==Copy the filtered dump to the new repo==
			Before we can add the revisions to the repo, we need to at least create the folder where the fitered dump files are in the new repo, or we will get an error:
			<pre>cd ~/palea_svn
			svn co http://palea.cgrb.oregonstate.edu:/svn/svn2git
			cd svn2git
			mkdir -p trunk/ontology/collaborators_ontology/gramene/traits
			svn add trunk
			svn commit --username=elserj
			</pre>
			Then back on palea (svn server):
			<pre>cd /data/svnrepos
			sudo svnadmin load ./svn2git < /lemma/justin/Poc_svn_migration/trait.obo_filtered_dump
			</pre>

			Fix the file path (from the svn client):
			<pre>svn up
			svn mv trunk/ontology/collaborators_ontology/gramene/traits/trait.obo ./
			svn up
			svn rm trunk
			svn commit</pre>


			At this point, we should have a fresh clean svn repo with only the filtered files in it with all revision info.

			==Copy the svn repo to github==
			Checkout the repo from github