Git/hacks

Search

git grep only works on the current content of your working copy. To search through all revisions, you can use something like

# -F is --fixed-strings
# Interpret PATTERN as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched.
git rev-list --all | ( while read revision; do git grep -F 'wgAWSCredentials' $revision; done; )

You can also search with git log to find commits of interest. Simply use the --grep option and add the -i option (short for --regexp-ignore-case) to make your pattern case-insensitive. Adding --reverse will show commits ordered from the beginning of time - so finding the earliest commit first.

git log --reverse --grep rocky -i

will show the commits from the start of the repository that have the word 'Rocky' in the commit message, without regard to case (ie. matching rocKy).

Similarly, you can use git log -S"word" or git log -G"word"

Show commits (in log format) that match the non-case-sensitive regex 'Rocky'

git log --reverse -G"Rocky" -i

Show the commit contents as a patch:

git log --reverse -G"Rocky" -i --patch

See https://stackoverflow.com/questions/1337320/how-can-i-grep-git-commits-for-a-certain-word for more details, and the section below on git log.

Ignore File Mode

Sometimes you have to temporarily change file modes (or some script might alter your working directory). Anyway, to ignore file mode changes temporarily, you can just add -c core.fileMode=false to your command. E.g.:

git -c core.fileMode=false status

You can also put this into a repo or global config, but you probably shouldn't.

If you need to change filemodes back to the way they were in the repo, you can do something like

git diff --summary | grep --color 'mode change 100755 => 100644' | cut -d' ' -f7 | xargs chmod +x
git diff --summary | grep --color 'mode change 100644 => 100755' | cut -d' ' -f7 | xargs chmod -x

Working on a remote terminal

If you're working on a remote terminal and your repository get's complicated, but you don't have desktop tools like meld or gitk to look at it, you can copy the remote repo to your desktop with rsync, even if you have to jump through a bastion host with something like:

rsync -e "ssh -t bastion ssh -A" -ravz centos@10.0.50.68:/opt/meza/ ./meza-es1/

If that complains about host-key verification, then simply do an SSH first to the host, accept the host key identity, and logout. Now the rsync will work because the host-key is already accepted as valid.

Tracking remote branches

Checkout a specific branch from origin, and track it. What if you enter git checkout -b REL1_29 when you *should have entered* git checkout -b REL1_29 origin/REL1_29? Just tell git that you meant to track the branch in origin: git branch --set-upstream-to=origin/REL1_29 REL1_29 Set upstream doesn't do what's intended here! You should delete the branch that you created as a 'copy'; and then checkout the branch correctly. To checkout the branch correctly, just IGNORE the -b flag altogether. If you do use the -b flag, then you must use the long form that references the remote branch to track. If you just do a 'git checkout foo' and there exists the same branch upstream at origin/foo then git will checkout a new branch set to track the remote.

Tracing in Status

Add GIT_TRACE=1 to your command to see more of what's going on. (Note that there are no spaces and no semicolon.)

GIT_TRACE=1 git status

Just the facts

Want simple red/green diff lines?

See all the changes in color, but without any context lines, and without the leading +/-/[space] This makes it easy to grab changes and stuff them in another file for example.

git diff -U0 --color myfile | sed -r "s/^([^-+ ]*)[-+ ]/\\1/"

What is this miscellaneous file, and how does it compare to what's in my repo?

Say you've got a config file lying around (untracked) in your local working tree. It's not in your current project branch, but you know it's an important file. How does it compare to what's in the freephile remote, es128 branch version of the file?

git diff freephile/es128:config/core/MezaLocalExtensions.yml config/core/MezaLocalExtensions.yml

How has this file changed over time?

While you can use gitk on your desktop, if you're on a headless server, you can ask git log to show you the patch for each commit: git log -p path/to/file

Compare a file between branches

Do you think a particular file has changed between two separate feature branches (ea based off master)?

# git diff branch1..branch2 -- path/to/file
git diff es128-rebased..get-to-know-meza -- manual/commands.md

Add that forgotten file

You forgot to add a file to the last commit? Just add it to the index, and commit with --amend. Added a file that shouldn't be there? git rm it. If you leave off the -m (message) option in the new commit, it will let you re-use the last commit message. This lets you "undo the last commit" and redo it right. You usually do not want to amend a commit if you've already pushed it to other repos, but if it's just local --amend is awesome-sauce.

git add forgotten.php
git rm oops.txt
git commit --amend
git log --stat

Undo a commit

Sometimes a commit is just wrong or 'breaks the build' so to speak. If you really just want to undo a commit, then

# view what changes were made in the last commit
git difftool HEAD~
# reset --soft will undo the last commit, but leave the changes locally as 'modified' files that you can re-work or commit.
git reset --soft HEAD~
# reset --hard will undo the last commit, and discard any changes that were made in the commit. Your work tree will be identical to the way it was after the prior commit.
git reset --hard HEAD~

Using reset you can even go backwards several commits if you want.

The entire internet tells you not to reset if you've already pushed. But if you're just pushing (from machine A: your desktop) to a remote (machine B: eg. GitHub) so that you can then pull those changes into another space (machine C: development/staging/production/other machine) you can push your changes with --force; and pull them from that other environment. It may be quicker and easier though to identify the SHA of the n-1 commit and then reset on machine C

git log -n 4
# find the commit SHA, and use it on machine C
git reset --hard d89c004cfe4bd67838fb41c7a6644bb15feee5cc

Credit: there's a clear explanation at https://www.git-tower.com/learn/git/faq/undo-last-commit

Good and Consistent commit messages

You hopefully know what constitutes a good commit message. What about consistency? Say you need to add standard language to every commit message like "This work was performed for NASA GRC-ATF by WikiWorks per NASA Contract NNC15BA02B."

To consistently format your commit messages, use the configuration parameter commit.template which you can do with git config, or else in your ~/.gitconfig file. Here's my commit.template

https://thoughtbot.com/blog/5-useful-tips-for-a-better-commit-message

The GitLab project uses the concept of commit templates

Git Branch

• list the branches you have locally git branch --list
• list the branches that are on remotes git branch -r --list
• checkout a remote branch, setting the local one to track git checkout -t freephile/patch-1 or simply git checkout patch-1 if push.autoSetupRemote is set properly
• git branch --merged (while on master) shows you branches which are fully contained in HEAD, and can be deleted.

Comparing local to upstream

If you do a git status and it tells you that your (local) branch is behind the remote tracking branch, then maybe you want to look at what those changes are.

# On branch master
# Your branch is behind 'freephile/master' by 6 commits, and can be fast-forwarded.
#   (use "git pull" to update your local branch)

Tell git to fetch the branch named 'master' from the remote named 'origin'. Git fetch will not affect the files in your working directory; it does not try to merge changes like git pull does.
git fetch origin master

When the remote branch is fetched, it can be referenced locally via FETCH_HEAD. Tell git to diff the working directory files against the FETCHed branch's HEAD and report the results in summary format.
git diff --summary FETCH_HEAD

If you want to see changes to a specific file, for example myfile.js, skip the --summary option and reference the file you want (or tree). Note: paths (or pathspecs) should be separated from command options with a double dash (--)
git diff FETCH_HEAD -- mydir/myfile.js

Comparing Branches

What's in this old branch, not in my new branch (ignoring merges)?

• git log oldbranch ^newbranch --no-merges
• git log master..feature (everything on the feature branch that is not in master)
• git log HEAD..freephile/master (everything in the remote 'freephile/master' that's not in HEAD locally)
• git log origin/master..HEAD (everything in the local branch that you would push)
• git log --left-right master...experiment (triple dot shows everything since the common ancestor)

How did I get here?

What branch did this branch originate from? Or more precisely, what's the nearest ancestor commit on a separate branch?

branch=`git rev-parse --abbrev-ref HEAD` \
git show-branch -a 2>/dev/null \
| grep '\*' \
| grep -v "$branch" \
| head -n1 \
| sed 's/.*\[\(.*\)\].*/\1/' \
| sed 's/[\^~].*//'

Explanation:

1. Grab the name of the current branch.
2. Show all commits (with errors or warnings going to dev null).
3. Ancestors of the current commit are indicated by a star. Filter out everything else.
4. Ignore all the commits in the current branch.
5. The first commit remaining is the nearest ancestor from another branch.
6. Branch names are displayed [in brackets]. Ignore the brackets and everything else.
7. Sometimes the branch name will include a ~2 or ^1 to indicate how many commits are between the referenced commit and the branch tip. We don't care. Ignore them.

Don't merge, Rebase!

Get familiar with how to rebase your work each day http://www.bitsnbites.eu/a-tidy-linear-git-history/

If you forget to pull before you commit some local changes, you might just be able to `git rebase` to pull and re-play your changes on top of the branch.

Delete local and remote merged branches

git branch --merged | egrep -v "(^\*|master|dev)" | xargs -I % echo 'git branch -d % ; git push --delete freephile % ;'

Using echo helps you look before you leap. % is the replacement string in xargs. Change 'echo' to 'sh -c' to execute. Does both local and remote prunes.

Git Log

What's the commit history? The man page for git log is so big, it's a forest. There are examples in the book for viewing commit history. For a quick cheatsheet, try these.

--stat gives a nice view of what happened in the log.

git log --stat
# try these other git log variatiations
git log --patch
git log -2
git log -p -2 # - p is the same as --patch
git log --pretty
git log --pretty=oneline
git log --pretty=short
git log --pretty=full
git log --pretty=fuller
git log --graph
git log --pretty=oneline --graph
git log --name-only
git log --name-status # show the file names and letter codes for the changes
# example
git log --left-right  --name-status origin/grc-atf...origin/main
git log --abbrev-commit
git log -S<string> # find when the number of instances of the string are added or deleted
git log -G<regex> # same with POSIX extended regex
git log -- filename # look for commits that touch filename
git log -p -- filename
git log --since 'last week'
git log --since 2017-01-01

How do you ignore a directory, but make an exception? What if you already added certain directories to git but want to stop tracking them now (ie. "take them out of version control")? A combination of editting your .gitignore file and git rm --cached to the rescue. I had accidentally added and committed some files into git which should have been ignored because they are 3rd party files managed by Composer. I fixed my .gitignore to track only what I want while ignoring a parent directory:

/nbproject/*
# ignore everything in the 'vendor' directory
/vendor/*
# but don't ignore the 'eqt' directory
!/vendor/eqt/

Then you simply remove all files from git's index, and add them back (only now adding them back will look to .gitignore for the corrected rules)

git rm -r --cached .
git add .
git commit -m 'ignoring vendor/*'

Tags

With git, you can just tag something with git tag foo. This produces a 'lightweight' tag . Use "annotated tags" whenever you want to know when something was tagged and who did it. Pass an empty message if you really don't care or need extra annotation.

git tag -am '' 'REL-1.0-alpha'

Git merge branch of another remote

When using git between a local repository and a single 'origin' remote, it's a simple process to work locally and push things back up to origin. But, what if you have a separate remote repository... perhaps on GitHub, or a collaborator who has similar sources but not using your origin (so disconnected, and perhaps not even linked ancestrally like a fork). How do you add that other remote to your project and then pull in the code "they" have on top of yours? Here's an example of how we started with a repo from github and added a repo that we were developing privately. (The reality is that we were developing a repo privately; created a sibling version of the code at github; and then wanted to re-incorporate the changes of the github repo back into our private repo.)

# start with one 'origin' remote
git clone https://github.com/freephile/qb.git
# add another remote that has similar code
git remote add private greg@eqt:/home/greg/src/ad.git
# check
git remote -v
# rename it for the host it came from
git remote rename private eqt
# pull our 'eqt' remote onto the master branch of the code we got from 'origin'
git pull eqt master
# But there are some unversioned files in the way (git complains)
# So, create a new 'dev' branch and stuff all the new things there (which we can either delete, or resume later)
git checkout -b dev
git add hosts package.json requirements.txt scripts/
git commit -m 'stashing some files into a dev branch'
git checkout master
# now we can merge again
git pull eqt master
# git complains about some merge conflicts (changes on both sides so it can't just do a fast-forward)
# edit those files to remove the conflict markers and get them the way you want them
git add install* launch.yml
# finish your merge by committing the result
git commit -m 'merged additional plays from eqt'
# and push upstream to 'origin'
git push
# and push to other remote 'eqt'
git push eqt

Unbloat

https://stackoverflow.com/questions/3797907/how-to-remove-unused-objects-from-a-git-repository

git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 \
  -c gc.rerereresolved=0 -c gc.rerereunresolved=0 \
  -c gc.pruneExpire=now gc

Put your project on GitHub

Been hacking away on a project and now it's time to unveil it? Here are the quick and easy steps to get your local repo into your GitHub account.

The short version is

1. Go to GitHub.com and create a new repo - but only create a README and/or LICENSE (the less the better).
2. Copy the HTTPS "clone" URL (or the SSH one if you have an authorized key on the remote host).
3. Go to your remote host where you have developed your code
4. git init your project if you haven't already. And also configure git with user.name and user.email
5. git add .
6. git commit -m 'First commit'
7. git remote add the URL from step 2
8. git pull --rebase
9. git push

# first make a "bare" clone
# from your server
git clone --bare greg@example.com:/var/www/baz.com/www/project
# or from a local directory
git clone --bare /tmp/project
# produces project.git directory
cd project.git
# push that to your new project (created via browser at GitHub.com)
git push --mirror https://github.com/USER/project.git
# go back to the original project folder
cd /tmp/project
# optionally remove local remotes that are no longer needed
git remote remove origin
# add GitHub as a remote named 'github' (assuming SSH key-based auth)
git remote add github git@github.com:USER/project.git
# pull in anything from 'upstream' (assuming that now GitHub is the canonical source)
git pull github master
# push any changes up; setting the new remote as the upstream for the 'master' branch
git push --set-upstream github master
# from now on you can just 'git push' from master

Create a README from a webpage

Now create a fine README file using pandoc to convert your webpage to Markdown

pandoc --standalone --read html https://freephile.org/barcode/index.html -o README.md

Remember your password

I know how to configure my user and email, but how do I tell git to remember my password for repos that I'm interacting with? The answer is credential helper

Rewriting History

Developers need git as a way to track changes and collaborate on software. Release Engineers need git, and higher-level tools to manage entire repositories of code. Whether Developer or Release Engineer, sometimes you need a do over.

When rewriting history in your git project, If an interactive git rebase using 'edit' or 'reword' doesn't suffice for your use-case, you are probably looking for a higher-level tool.

To make subtle changes, or even large changes on a whole series of commits in a git repository, git filter-repo(Python) is that tool. The git maintainers themselves do NOT recommend using git filter-branch - which is slow and painful and prone to problems. Consult the Git Filter Repo User Manual for examples.

HTTP Secure = HTTP using SSL or the newer TLS

Uniform Resource Locator = an address of a web page or other component, including the protocol.