Git/migrating to git

Simple

For simple, small migrations, follow the process described by CollabNet in their blog (and elsewhere). For anything else, don't do it! git svn clone is not a migration tool.

Summary

A summary of the steps for migrating your version control system to git from subversion

1. Discussions with stakeholders, project lead
2. Leverage the resources and expertise of prior large migrations including Pro Git 2nd Ed. Eclipse Foundation, Atlassian, Wikimedia Foundation, EclipseCon proposed talk by Max Anderson, Drupal, PostgreSQL, and Pentaho. Be sure to include the "before", the migration itself, and the "after" migration work.
3. Understand the concept of Git repos
4. Know the caveats
5. Plan and structure your Git space
6. Decide what to do with your existing code
   1. Archive your current SVN repository?
   2. Import your history into git?
7. Do the migration.
   1. Map users
   2. Migrations must include at least the following details
      1. Migration timeline
      2. mapping of current code to new Git repos
      3. decision regarding existing code (archive or import)
      4. A description for each repository (which will be visible in the web view)
   3. Use scripted recipes for LARGE migrations
8. Importing your SVN history into Git
   1. If you can migrate using git svn clone which is a tool providing a bi-directional conduit of changesets between subversion and git, then good for you! Your project is small and uncomplicated. For larger, more complicated migrations, this tool is not suited for the job. It will take too long, and simply will fail to produce a git repository. I don't understand why Atlassian recommends this approach in their "tutorial" without telling you that it will fail; or at least providing the major caveats. Still, you can read up on the simplistic scenario
   2. Using svn2git
   3. Using reposurgeon - a tool by Eric Raymond
9. Convert svn:ignore properties to .gitignore file (example of why you need to later delete empty commits which reflect properties not code changes)
10. Verification

    mkdir -p /tmp/verify
    cd /tmp/verify
    for v in 2.0.0 2.1.0 2.2.0 ; do
    svn co $v
    done
    
    # export git tags
    cd /tmp/conv/myproj-git
    for v in 2.0.0 2.1.0 2.2.0 ; do
      git archive --format=tar --prefix myproj-$v/ v$v | gzip \
        /tmp/verify/git/myproj-$v-git.tar.gz
    
    done
    cd /tmp/verify/git
    for v in 2.0.0 2.1.0 2.2.0 ; do
      tar xvzf myproj-$v-git.tar.gz
    done
    
    # compare them
    diff -ur /tmp/verify /tmp/verify/git
    

11. Build
12. Service integrations
    1. JIRA / Issue Tracker
    2. ReviewBoard / Code Review
    3. Jenkins / Build system
    4. SQA
    5. OpenGrok / Code indexing browser
13. Client and End User configurations
    1. Create keys, add each to client and server
    2. Install / setup TortoiseGit for Windows
    3. Add EGit to Eclipse
    4. Netbeans natively supports Git since v7.1
14. Repository permissions and group definitions, key imports
15. Establish Git Resources
16. Create Git Task Force
17. Update references in your literature, project documents, websites, systems, reference materials and procedure documents to reference the new systems. This step can be ameliorated if in the beginning you reference code in a generic way such as "code.example.com" where you can then link to various aspects and implementations of your code systems; rather than naming them specifically based on technology or implementation.

Post-processing

• Use the BFG Repo cleaner to remove large files, passwords, unwanted paths
• clone it to reduce size git clone file:///path/to/repo

Lessons Learned

• https://techbase.kde.org/Projects/MovetoGit
• http://blog.smartbear.com/software-quality/migrating-from-subversion-to-git-lessons-learned/

• http://www.midwesternmac.com/blogs/jeff-geerling/switching-svn-repository-svn2git

Caveats

A single Subversion repository almost always contains multiple "projects" - each with it's own 'trunk', 'branches' and 'tags'. One thing you'll find moving from SVN to Git is that Git repositories aren't designed for multiple projects in the way that SVN is used. All the separate projects (from a single SVN repository) should be migrated to separate Git repositories. A tag or branch in a Git repository is always global to the repository -- so split the code into repositories along boundaries that make sense semantically. (Note that git has much better support for including library code into a project. The feature is called "sub modules". Thus library code can be semantically split out into it's own repository, and that library can be re-used across multiple git repositories. This is like svn "externals" only better.

Upside

Once you've established a git infrastructure for version control, git to git migrations are incredibly easy... at least for the core git repository functions. Just add another remote to push/pull. This means that if you wish to change your git infrastructure to use a different system, the work involved will mostly be about the extra features bundled with the system (e.g. web viewer, code review, etc.) and integrations.

Combining git repos

You might desire to reorganize your code in the migration process. There are several tools which allow you to merge git repositories together.

• http://search.cpan.org/dist/Git-FastExport/script/git-stitch-repo git-stitch-repo] is made for linear repos
• This other one by the same name, authored by Philippe Bruhat (BooK) is nonetheless capable of merging two (or more) repositories
• This one, git-merge-repos is interesting because it talks about taking multiple repositories with more or less the same branches or tags, and merging them at the tag
• This post on Stackoverflow about combining multiple git repositories mentions git-stitch-repo, and also how it gained the capability to work with non-linear merge histories. It also explains how to do repo merges with git-filter-branch. Note that git-filter-branch requires you to rewrite your history (breaking SHA1 sums).
• The subtree merge strategy page on kernel.org shows you how to do this.
• https://westmarch.j5int.com/2014/06/splicing-git-repositories-together/ Splice Repos is a python script. It's more recent than some others and a better tool because it's based on fast-export/fast-import . It grew out of the JBossTools git migration (which itself has some useful info on procedures).
• Also, the reposurgeon tool itself can assist you with the re-organization of your sources.

Submodules

Sometimes, 'combining' your work with other work is best accomplished through submodules. Git submodules are a way for you to store other repositories in directories of your project. This is most often used to handle 'vendor' code, or libraries. However, submodules can be used whenever you want to combine repositories, yet maintain them independently.