Open main menu
Freephile Wiki

Git

Revision as of 20:59, 5 June 2018 by Freephile (talk | contribs) (close list)

The Git system (http://git-scm.com/) was originally developed by Linus Torvalds as a distributed version control system to replace BitKeeper for the Linux Kernel project

The bottom line is this: modern software development may take many "forms", but it usually boils down to this. Git enables such a branched workflow. That is why distributed version control in general, and git in particular, is the most widely adopted version control system for software development [1]

See Git/hacks for example commands

Intro to Git

Greg Rundlett setup git repository hosting services at the IIC (http://svn.iic.harvard.edu/gitweb/)

Why do we have git? Because Git is better than X Now that we have the "My DVCS is better than your DVCS" argument out of the way, you can actually get some valuable insights from that website if you are interested in comparing Git with Mercurial, Bazaar, Subversion or Perforce. If I had to single out one primary advantage of Git, it would be that it actually features branching and merging.

Repo visibility is completely customizable, as are individual permissions to write to repos. I've installed a system called gitosis to handle the privileges through a special git repository. It uses Public Key cryptography rather than granting SSH accounts to everyone. This makes it really easy to do your work securely without even needing a password. For the curious, the actual mechanics of gitosis are detailed at scie.nti.st. Although that link may still work, there is also https://git-scm.com/book/en/v1/Git-on-the-Server-Gitosis.

You can actually host your own private repository on any server that you have access to. To collaborate with your peers, export a bare repo [2] and use SSH keys. more

To host a public repo, go to http://github.com

You might want to read about how we migrate from Subversion to Git or how others have done it

Initial Configuration

In your first time setting up Git on a new computer, you want to configure your username and email among other settings. 

git config --global user.name "Greg Rundlett"
git config --global user.email greg@freephile.com
git config --global core.editor vim
git config --global diff.tool meld
git config --global --add color.ui true
# recent packaged versions of git might be 1.8.3.1 (CentOS 7.4) whereas the current available is 2.15.1
# push.default is unset; its implicit value is changing in
# Git 2.0 from 'matching' to 'simple'. To squelch this message
# and adopt the new behavior now, use:
git config --global push.default simple

Example .gitconfig

git config --global -e and paste in the following: 

[user]
        name = Greg Rundlett
        email = greg@freephile.com

[alias]
        co = checkout
        ci = commit
        unstage = reset HEAD
        wu = log --stat origin..@{0}
        wup = log -p origin..@{0}
        lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative

[core]
        excludesfile = /home/greg/.gitignore_global
        editor = vim

[color]
        ui = true

[credential]
        helper = cache --timeout=3600

# ------------------ M E R G E -------------------------
[merge]
    tool = meld

[mergetool "meld"]
    cmd = meld --auto-merge \"$LOCAL\" \"$BASE\" \"$REMOTE\" --output \"$MERGED\" --label \"MERGE (REMOTE BASE MY)\"
    trustExitCode = false

[mergetool]
    # don't ask if we want to skip merge
    prompt = false

    # don't create backup *.orig files
    keepBackup = false

# ------------------ D I F F -------------------------
[diff]
    guitool = meld

[difftool "meld"]
    cmd = meld \"$LOCAL\" \"$REMOTE\" --label \"DIFF (Original v. Mine)\"


[push]
        default = simple
[gitreview]
        remote = origin
[help]
        format = web
[web]
#       browser = google-chrome
        browser = firefox

If you want to browse the git help in your browser (as above), then make sure you sudo apt-get install git-doc to get the HTML files


Decentralized Workflow and Branching Model

Here's a good explanation on how most groups use Git in a successful branching model for development, "master", release branches, hotfixes, etc. http://nvie.com/posts/a-successful-git-branching-model/


Tools

Meza Big Picture

Reporting

How many lines have I contributed since last year? 

git log --stat --author $(git config --get user.email) --since="last year" --until="last month" | awk -F',' '/files? changed/ {
    files += $1
    insertions += $2
    deletions += $3
    print
}
END {
    print "Files Changed: " files
    print "Insertions: " insertions
    print "Deletions: " deletions
    print "Lines changed: " insertions + deletions

}'

Visualizing

There are a lot of ways to use the git log command to visualize your history. Everything from git log --pretty=oneline or git log --graph --oneline to git log --graph --full-history --all --color --date=short --pretty=format:"%x1b[31m%h%x09%x1b[32m%d%x1b[0m%x20%ad %s" which you probably want to put into your ~/.gitconfig as some alias because who the heck is going to remember that one?[3] Of course there are also many GUI clients.

Documentation

Git and Subversion

Git Repo Hosting

It seems everyone uses GitHub these days (~8 million users, 20 million projects). It's not a bad solution, but there are some drawbacks (you might not want Microsoft looking at your private code), and whether you're just looking to host your own project or setup enterprise-wide git hosting, there are other ways to host your projects in Git. Let's take a look at some options. You can see a comparison of some of these options at https://ethercalc.org/choose-the-git-host.html

Copyleft

  1. Pagure is a forge written in Python. With pagure you can host your project with its documentation, let your users report issues or request enhancements using the ticketing system and build your community of contributors by allowing them to fork your projects and contribute to it via the now-popular pull-request mechanism. Pagure is a RedHat project, licensed under the GPL v2. Pagure uses gitolite for controlling user access to repositories. At the time of this writing, Pagure is the project eQuality Technology prefers. Some notable features:
    • customizations can be done out of tree which makes upgrading fairly painless
    • entire projects can be easily exported and imported from one instance to another, all by using only git, including issues, pull requests, and their comments because they are stored as git repos - this means for example one could host their project on pagure.io for some time and then migrate everything to their own instance once they decide to self-host - the project can even be mirrored on multiple servers with everything perpetually synchronized by external cron tasks or pagure hooks.
  2. Coding Team is a forge written in PHP using the AGPL license.

    CodingTeam is a free (as in freedom) forge written in PHP with a lot of collaborative work and communication tools. This is a free software released under the GNU Affero General Public License version 3. A little bit of history: the project is born in march 2005 but the first public version (0.42) was released in june 2007. The first motivation of this project was to make a good forge for French-speaking users. Now, developers focus on accessibility and ease-of-use (this forge is recognized by many people as one of the only forge to have understandable and intuitive HCI). CodingTeam is available in many languages and the "lightweight but powerful" philosophy now benefits to everybody!

    Sadly, the project doesn't seem to have progressed much past the initial support for git and mercurial.

  3. Kallithea is a fork of the Rhode Code project (after they made disturbing changes to their license) maintained by the Software Freedom Conservancy.

    a member project of Software Freedom Conservancy, is a GPLv3'd, Free Software source code management system that supports two leading version control systems, Mercurial and Git, and has a web interface that is easy to use for users and admins. You can install Kallithea on your own server and host repositories for the version control system of your choice.

  4. gitolite is similar to gitosis, but adds features; is currently maintained; and used by large installations like Kernel.org and KDE.
  5. Gitosis was created by Garry Dolley (up the irons) back in 2007, and is now maintained by Tommi Virtanen (Tv). Gitosis can still work well for small dev groups who want to use key-based auth for their code like I setup at Harvard' IIC. See https://git-scm.com/book/en/v1/Git-on-the-Server-Gitosis
  6. Gitorious code is available, but the AGPL project was bought out by GitLab, and hasn't been developed since 2015

Not Copyleft

  1. Phabricator is a full forge; and is used by the WMF. However, the code is Apache licensed, not GPL.
  2. GitHub can make it a bit easier for developers who are used to working with svn because they have 'built-in' svn client support.[4] GitHub Enterprise can be installed on your own servers. GitHub costs money for private repos.
  3. GitLab is the "popular" alternative to GitHub that comes in "Community", "Enterprise" and "Cloud" versions. They say it's "better than GitHub" One nice thing that I note is that they've integrated git-annex for large binary support. This is a nice plus. (I've personally used git-annex as a file backup system -- like DropBox. In my case, it was to make backups of UbuntuOne when that service was discontinued.) You can install GitLab in minutes on your own server. It's costs if you want the Enterprise version.
  4. GitBlit is "a pure Java git server that just works". It's run by James Moger and scores of other contributors. GitBlit is self-hosted, so you can get a feel for it quickly by looking at the projects own source code. It does have some nice features. Wikimedia Ops used Gitblit prior to switching to Phabricator. GitBlit is designed primarily as a tool for small workgroups who want to host centralized repositories. It comes in two options: GO is an integrated single stack solution so you can really get up and running in no time flat; and there is a WAR version (assumes you already have a servlet container like Jetty or Tomcat.) Gitblit requires a Java 7 Runtime Environment (JRE) Gitblit comes with a lot of features
  5. Gitea is a community managed fork of Gogs, lightweight code hosting solution written in Go and published under the MIT license. It's self-hosted. The Gogs project is/was controlled by a single author, and the Gitea fork happened in 2016 by several users and contributors who wanted a different project management style: community-driven. Gitea is librejs compatible.
  6. Notabug offers free public hosting, and is based on a modified version of Gogs. Despite the leadership of Peers, it's still MIT-licensed.

Background

I got into Git to make managing Drupal projects possible. Now that we have a distributed VCS, we can actually collaborate in multiple ways (more than just a centralized model[5]), and we can work offline . We can easily branch and merge. We can pull in updates from the Drupal core maintainers. We can pull in updates from the Drupal community (module maintainers). We can track our own custom modules - especially if they are developed by more than one person. We can branch, tag and deploy these combined sources to multiple instances (dev, staging, production) across multiple instances of Drupal (e.g. client projects that should share the exact same codebase) and have complete clarity on where each project stands regarding patches and security fixes or updates. There have been several attempts in the Drupal community (credit to Moshe, Ben,[6] Dries, et al.[7]) to get this all working over the past year or so and I think we're finally seeing the tide turn in our favor. Drupal may actually start to be "easy" to maintain and deploy. Actually, the whole point is to BE up-to-date on all our projects. This only happens when it is feasible. Git makes it feasible.

I pulled down the whole clone (which is only 26MB in size) of mikl's public history of Drupal in a few minutes.

With Git, it is even possible to manage projects which are composites of multiple sources [8] (Ever tried that with Subversion's 'externals'? -- Don't bother to try it with Subversion because it just doesn't work! Not only are externals feature poor, you'll still end up wanting to merge even more than before. And as we know, merging is not a feature in Subversion.)


Comparisons of Branches, sources etc.

When comparing large trees of files, you need a good set of tools to be able to filter out noise. Normally KDiff3 is very good at this. However, KDiff3 is not available yet as a package for Ubuntu. If you compile it from source, I still found that the most important feature -- the preprocessor -- would not work if more than one command were piped together.

Specifically, I couldn't use this preprocessor setting which worked great for me in the past: sed "s/\$\(Id\|Revision\|Author\|Log\|Header\|Date\).*\$/\$\1\$/" | sed "s#/\* vim: set.*#/* vim: set ... */#" | sed "s/^.*opyright.*$/Copyright Greg Rundlett :-)/"

Without the benefit of a good preprocessor, KDiff would falsely tell me that there were 1,523 files which differed between DRUPAL-6-9 and DRUPAL-6-10

Kompare seemed hard to configure, and Meld didn't apply the pre-processor to directory compares. So, I used the command-line version of diff:
greg@greg-linux:~/work/git/drupal$ diff -r --exclude=.git --exclude=.svn --brief --ignore-matching-lines="$Id" . /var/www/drupal/ 

Only in /var/www/drupal/: favicon.ico
Only in /var/www/drupal/sites/all: modules
Only in /var/www/drupal/sites/all: themes
Only in /var/www/drupal/sites: dev.example.org
Only in /var/www/drupal/sites: example.org
Only in /var/www/drupal/sites: localhost
Only in /var/www/drupal/sites: prod.example.org
Only in /var/www/drupal/sites: www.example.org

Building a Git server

(This recipe was from 2008. It may still work, but ymmv)

Generally following the series of instructions at http://vafer.org/blog/20080115011413 I installed Git-core, gitweb and gitosis on the host 

# install git, the git-svn tool and the viewvc equivalent
apt-get install git-core git-svn gitweb

# run the git-daemon - a really simple server for git repositories
sudo -u git git-daemon --reuseaddr --verbose --base-path=/home/git/repositories/ --detach

# create a service that starts when the machine starts
cat > /etc/init.d/git-daemon << EOF
#!/bin/sh

test -f /usr/bin/git-daemon || exit 0

. /lib/lsb/init-functions

GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach"

case "$1" in
start)  log_daemon_msg "Starting git-daemon"

start-stop-daemon --start -c git:git --quiet --background \
--exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS}

log_end_msg $?
;;
stop)   log_daemon_msg "Stopping git-daemon"

start-stop-daemon --stop --quiet --name git-daemon

log_end_msg $?
;;
*)      log_action_msg "Usage: /etc/init.d/git-daemon {start|stop}"
exit 2
;;
esac
exit 0
EOF

# and add it to the various run-levels
update-rc.d git-daemon defaults

# setup the configuration of the gitweb repository browser
vi /etc/gitweb.conf

# Add cgi capabilities to our "Version Control" host
vi /etc/apache2/sites-available/git

# check the syntax of our apache configuration
apache2ctl configtest

# oops, the mod_rewrite isn't enabled
a2enmod rewrite

# check syntax and errors again
apache2ctl configtest

# restart apache
/etc/init.d/apache2 restart

# see what may have been added to the web docroot
ls /var/www/

# check to see that gitweb is installed
# and see what files it installs since they aren't in /var/www
dpkg --status gitweb
dpkg --listfiles gitweb

# create a symbolic link between the gitweb software and the repository root where we'll be viewing it.
ln -s /usr/share/gitweb /home/git/

# see what we have so far
ls /home/git/

# setup our gitweb configuration
vi /etc/gitweb.conf

# check if gitosis is packaged for ubuntu
apt-cache search gitosis
# and install it
apt-get install gitosis


Rant

quote 
After replacing the 1.0 code with 1.1 code, svn status will show files with local modifications as well as, perhaps, some unversioned files. If we did what we were supposed to do, the unversioned files are only those new files introduced in the 1.1 release of libcomplex--we run svn add on those to get them under version control. If the 1.1 code no longer has certain files that were in the 1.0 tree, it may be hard to notice them; you'd have to compare the two trees with some external tool and then svn delete any files present in 1.0 but not in 1.1. (Although it might also be just fine to let these same files live on in unused obscurity!) Finally, once our current working copy contains only the libcomplex 1.1 code, we commit the changes we made to get it looking that way. [9]

So, according to the official Subversion manual, the right way to track source code that you build on, you should

  1. download an export (they don't even consider that your vendor should be and IS using a VCS)
  2. add stuff that they added
  3. launch external tools to find out if there were deletions
  4. or just ignore deletions (tell me why we're using source code managment tools?)
  5. commit those changes

They admit that this is not exactly what you would like: "...things can quickly degrade into a situation where we have to manually re-create our customizations in the new version."

The official manual continues: "Vendor drops that contain more than a few deletes, additions, and moves complicate the process of upgrading to each successive version of the third-party data. So Subversion supplies the svn_load_dirs.pl script to assist with this process."

Except that svn_load_dirs.pl does not come with Subversion [10], and even if it did, it has limited or zero capabilities to handle and TRACK the status, history, and provenance of your source code. In any situation where a conflict arises, svn_load_dirs.pl will ask YOU to figure it out -- without any supporting details. Despite what the quote from the manual implies, the actual README with svn_load_dirs.pl says that it should NOT be used with sources that come from another VCS... those imports should be scripted ala cvs2svn etc.

I don't recommend looking at these, you'll just learn how NOT to do things. But here are examples of others who have needlessly fought with Subversion to manage their Drupal sites and source code

References

  1. ↑ https://ianskerrett.wordpress.com/2014/06/23/eclipse-community-survey-2014-results/
  2. ↑ http://sitaramc.github.com/concepts-and-tips/bare.html
  3. ↑ https://stackoverflow.com/questions/1838873/visualizing-branch-topology-in-git
  4. ↑ Anyone can use git svn and all the other built-in git subversion bridging commands, GitHub simplifies how to do it.
  5. ↑ http://whygitisbetterthanx.com/#any-workflow
  6. ↑ http://drupal.org/node/197666
  7. ↑ http://drupal.org/node/355154
  8. ↑ http://git.or.cz/gitwiki/GitSubmoduleTutorial
  9. ↑ http://svnbook.red-bean.com/nightly/en/svn.advanced.vendorbr.html
  10. ↑ svn_load_dirs.pl had licensing questions and was removed from the project. Now I guess it's back, in a contrib section, not in a default install. Anyway, a newer, better set of options exist named collectively vcs-load-dirs
Retrieved from "https://wiki.freephile.org/wiki/index.php?title=Git&oldid=9024"