Difference between revisions of "Etckeeper"

From Freephile Wiki
Jump to navigation Jump to search
(use explicit paths in example)
(Adds commentary about hard-linked files)
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== Intro ==
 
== Intro ==
Etckeeper is a great tool created by Joey Hess to use your favorite VCS to keep track of what's going on in <code>/etc</code>.
+
[https://etckeeper.branchable.com/ Etckeeper] (also at https://github.com/joeyh/etckeeper) is a great tool created by Joey Hess to use your favorite VCS to keep track of what's going on in <code>/etc</code>.
 +
 
 +
== TLDR ==
 +
<source lang="bash">
 +
sudo su -
 +
apt-get install etckeeper
 +
cd /etc/
 +
# change VCS to 'git' by commenting out bzr; and uncomment git
 +
vim etckeeper/etckeeper.conf
 +
# add the contents below to etckeeper/commit.d/20mirror-outside-files
 +
# and make it executable
 +
chmod a+x etckeeper/commit.d/20mirror-outside-files
 +
# create your repo
 +
etckeeper init
 +
# commit your files
 +
etckeeper commit
 +
# now everything should just happen automagically anytime something in etc changes
 +
</source>
  
 
== Extending etckeeper ==
 
== Extending etckeeper ==
 
Say you want to keep track of changes to the configuration directory or file of some app.  You also want to make it automatic and painless.  Etckeeper leverages the commit hook in git to create that mirror.  So, for example, the most critical aspect of your [[wiki]] is the <code>LocalSettings.php</code> configuration file.  You don't want to commit the file to the version control of the project because it would expose sensitive data. etckeeper can be used to track changes to any file on the filesystem, but it's done discretely on the host.
 
Say you want to keep track of changes to the configuration directory or file of some app.  You also want to make it automatic and painless.  Etckeeper leverages the commit hook in git to create that mirror.  So, for example, the most critical aspect of your [[wiki]] is the <code>LocalSettings.php</code> configuration file.  You don't want to commit the file to the version control of the project because it would expose sensitive data. etckeeper can be used to track changes to any file on the filesystem, but it's done discretely on the host.
  
 +
git repos in the directories you want to track can cause problems.  One useful addition is to filter out anything found in '.gitignore' per the discussion at https://stackoverflow.com/questions/13713101/rsync-exclude-according-to-gitignore-hgignore-svnignore-like-filter-c  Or, since this script is just using rsync, you can add your own special cases as rsync calls
  
  
Line 30: Line 48:
 
   echo "  $LOCAL_PATH"
 
   echo "  $LOCAL_PATH"
 
   mkdir -p $MIRROR_ROOT/$LOCAL_PATH
 
   mkdir -p $MIRROR_ROOT/$LOCAL_PATH
   rsync -a $LOCAL_PATH/ $MIRROR_ROOT/$LOCAL_PATH
+
   rsync -a --filter=':- .gitignore' $LOCAL_PATH/ $MIRROR_ROOT/$LOCAL_PATH
 
}
 
}
  
Line 40: Line 58:
 
   rsync -a $LOCAL_PATH $MIRROR_ROOT/$DIRPATH
 
   rsync -a $LOCAL_PATH $MIRROR_ROOT/$DIRPATH
 
}
 
}
 +
 +
# special case where we don't want to mirror a sub-directory
 +
# we could also add a dummy .gitignore to the 'bin' directory
 +
rsync -a --exclude=ansible/ /home/greg/bin/ $MIRROR_ROOT/home/greg/bin/
  
 
###########################################
 
###########################################
Line 47: Line 69:
 
mirror_file "/var/www/html/wiki/LocalSettings.php"
 
mirror_file "/var/www/html/wiki/LocalSettings.php"
 
mirror_dir "/home/greg/data"
 
mirror_dir "/home/greg/data"
mirror_dir "/home/greg/bin"
+
## Special case handled above
 +
## mirror_dir "/home/greg/bin"
 +
</source>
 +
 
 +
== Problems with Hard-linked files ==
 +
 
 +
I got this with a recent <code>yum install</code>
 +
 
 +
<pre>
 +
etckeeper warning: hardlinked files could cause problems with git:
 +
./fail2ban/action.d/badips.pyc
 +
./fail2ban/action.d/badips.pyo
 +
./fail2ban/action.d/smtp.pyc
 +
./fail2ban/action.d/smtp.pyo
 +
 
 +
</pre>
 +
 
 +
I looked at the .gitignore in /etc and .pyo and .pyc files are already ignored, so there is no problem. In fact, you can verify that git is NOT tracking these files in etc with the following:
 +
 
 +
<source lang="bash">
 +
git ls-files fail2ban | grep py
 +
</source>
 +
 
 +
and compare to actual directory contents <code>ls -al /etc/fail2ban</code>
 +
 
 +
<source lang="bash">
 +
 
 +
# What files am I intentionally ignoring with .gitignore?
 +
 
 +
git ls-files --other --ignored --exclude-standard
 +
 
 +
# same as
 +
 
 +
git status --ignored
 +
 
 
</source>
 
</source>
  
Line 53: Line 109:
 
[[Category:Version Control]]
 
[[Category:Version Control]]
 
[[Category:DevOps]]
 
[[Category:DevOps]]
 
 
[[Category:VCS]]
 
[[Category:VCS]]
[[Category:Version Control]]
 

Latest revision as of 13:12, 28 August 2018

Intro[edit | edit source]

Etckeeper (also at https://github.com/joeyh/etckeeper) is a great tool created by Joey Hess to use your favorite VCS to keep track of what's going on in /etc.

TLDR[edit | edit source]

sudo su -
apt-get install etckeeper
cd /etc/
# change VCS to 'git' by commenting out bzr; and uncomment git
vim etckeeper/etckeeper.conf
# add the contents below to etckeeper/commit.d/20mirror-outside-files
# and make it executable
chmod a+x etckeeper/commit.d/20mirror-outside-files
# create your repo
etckeeper init
# commit your files
etckeeper commit
# now everything should just happen automagically anytime something in etc changes

Extending etckeeper[edit | edit source]

Say you want to keep track of changes to the configuration directory or file of some app. You also want to make it automatic and painless. Etckeeper leverages the commit hook in git to create that mirror. So, for example, the most critical aspect of your wiki is the LocalSettings.php configuration file. You don't want to commit the file to the version control of the project because it would expose sensitive data. etckeeper can be used to track changes to any file on the filesystem, but it's done discretely on the host.

git repos in the directories you want to track can cause problems. One useful addition is to filter out anything found in '.gitignore' per the discussion at https://stackoverflow.com/questions/13713101/rsync-exclude-according-to-gitignore-hgignore-svnignore-like-filter-c Or, since this script is just using rsync, you can add your own special cases as rsync calls


vi /etc/etckeeper/commit.d/20mirror-outside-files

#!/bin/sh
set -e

# Greg Rundlett info@equality-tech.com
# based on code from http://serverfault.com/questions/211425

# If you want other configuration data or files on the system also
# opportunistically tracked via etckeeper, use this script to copy them in.

# If there is a hook of some sort available related to the files
# you're mirroring, (e.g. Apache restart) 
# you can call etckeeper directly and track them
# proactively, rather than just opportunistically here.

MIRROR_ROOT=/etc/etckeeper.mirror.d
echo "etckeeper: mirroring outside files to $MIRROR_ROOT/:"

mirror_dir() {
   LOCAL_PATH=$1
   echo "  $LOCAL_PATH"
   mkdir -p $MIRROR_ROOT/$LOCAL_PATH
   rsync -a --filter=':- .gitignore' $LOCAL_PATH/ $MIRROR_ROOT/$LOCAL_PATH
}

mirror_file() {
   LOCAL_PATH=$1
   DIRPATH=`dirname $LOCAL_PATH`
   echo "  $LOCAL_PATH"
   mkdir -p $MIRROR_ROOT/$DIRPATH
   rsync -a $LOCAL_PATH $MIRROR_ROOT/$DIRPATH
}

# special case where we don't want to mirror a sub-directory
# we could also add a dummy .gitignore to the 'bin' directory
rsync -a --exclude=ansible/ /home/greg/bin/ $MIRROR_ROOT/home/greg/bin/

###########################################
## ADD lines below to invoke the system ###
###########################################

mirror_file "/var/www/html/wiki/LocalSettings.php"
mirror_dir "/home/greg/data"
## Special case handled above
## mirror_dir "/home/greg/bin"

Problems with Hard-linked files[edit | edit source]

I got this with a recent yum install

etckeeper warning: hardlinked files could cause problems with git: 
./fail2ban/action.d/badips.pyc
./fail2ban/action.d/badips.pyo
./fail2ban/action.d/smtp.pyc
./fail2ban/action.d/smtp.pyo

I looked at the .gitignore in /etc and .pyo and .pyc files are already ignored, so there is no problem. In fact, you can verify that git is NOT tracking these files in etc with the following:

git ls-files fail2ban | grep py

and compare to actual directory contents ls -al /etc/fail2ban

# What files am I intentionally ignoring with .gitignore?

git ls-files --other --ignored --exclude-standard

# same as

git status --ignored