Bash: Difference between revisions
mNo edit summary |
m Text replacement - "<(\/?)source" to "<$1syntaxhighlight" |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
This article is a cheat sheet for things you learn to do in Bash (The Bourne Again Shell). | This article is a cheat sheet for things you learn to do in Bash (The Bourne Again Shell). See also: the page of [[one-liners]]. | ||
== For Loops == | == For Loops == | ||
The bash one-liner for doing a for loop looks something like this: | The bash one-liner for doing a for loop looks something like this: | ||
< | <syntaxhighlight lang="bash"> | ||
for FILE in $(ls); do [COMMAND]; done | for FILE in $(ls); do [COMMAND]; done | ||
</ | </syntaxhighlight> | ||
Here is a real example that will lowercase the names of all files in the current directory: | Here is a real example that will lowercase the names of all files in the current directory: | ||
< | <syntaxhighlight lang="bash"> | ||
for FILE in $(ls); do mv $FILE $(echo $FILE | tr [A-Z] [a-z]); done | for FILE in $(ls); do mv $FILE $(echo $FILE | tr [A-Z] [a-z]); done | ||
</ | </syntaxhighlight> | ||
To do simple range looping, use the '''seq''' command: | To do simple range looping, use the '''seq''' command: | ||
< | <syntaxhighlight lang="bash"> | ||
for i in `seq 1 10`; do echo "$i, "; done | for i in `seq 1 10`; do echo "$i, "; done | ||
</ | </syntaxhighlight> | ||
== Arrays == | == Arrays == | ||
| Line 20: | Line 20: | ||
Simple arrays work like the following: | Simple arrays work like the following: | ||
< | <syntaxhighlight lang="bash"> | ||
#!/bin/bash | #!/bin/bash | ||
| Line 57: | Line 57: | ||
echo "we have ${#vegetables[@]} vegetables available" | echo "we have ${#vegetables[@]} vegetables available" | ||
printf "%s\n" "${vegetables[@]}" | printf "%s\n" "${vegetables[@]}" | ||
</ | </syntaxhighlight> | ||
== If construct == | == If construct == | ||
The <code>then</code> can go on the same line as the <code>if</code> as long as you use a semi-colon to terminate the if clause. Alternately, you can put the <code>then</code> on it's own line | The <code>then</code> can go on the same line as the <code>if</code> as long as you use a semi-colon to terminate the if clause. Alternately, you can put the <code>then</code> on it's own line | ||
< | <syntaxhighlight lang="bash"> | ||
if EXPR; then | if EXPR; then | ||
# do stuff | # do stuff | ||
fi | fi | ||
</ | </syntaxhighlight> | ||
is equivalent to | is equivalent to | ||
< | <syntaxhighlight lang="bash"> | ||
if EXPR | if EXPR | ||
then | then | ||
# do stuff | # do stuff | ||
fi | fi | ||
</ | </syntaxhighlight> | ||
Adding an <code>else</code> clause | Adding an <code>else</code> clause | ||
< | <syntaxhighlight lang="bash"> | ||
if EXPR; then | if EXPR; then | ||
# do stuff | # do stuff | ||
| Line 81: | Line 81: | ||
# do other stuff | # do other stuff | ||
fi | fi | ||
</ | </syntaxhighlight> | ||
Adding multiple <code>else</code> clauses with <code>elif; then</code> | Adding multiple <code>else</code> clauses with <code>elif; then</code> | ||
< | <syntaxhighlight lang="bash"> | ||
if EXPR; then | if EXPR; then | ||
# do stuff | # do stuff | ||
| Line 92: | Line 92: | ||
# final else | # final else | ||
fi | fi | ||
</ | </syntaxhighlight> | ||
Note: sometimes you want to comment out a section of an if/else block, or maybe it does nothing at all. In this case, you'll get an error. To avoid the error, you can use the [http://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.html bash built-in] <code>:</code> (colon command) | Note: sometimes you want to comment out a section of an if/else block, or maybe it does nothing at all. In this case, you'll get an error. To avoid the error, you can use the [http://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.html bash built-in] <code>:</code> (colon command) | ||
| Line 101: | Line 101: | ||
Do nothing beyond expanding arguments and performing redirections. The return status is zero. | Do nothing beyond expanding arguments and performing redirections. The return status is zero. | ||
< | <syntaxhighlight lang="bash"> | ||
if [ -f "/tmp/Non-existing-file.txt" ] ; then | if [ -f "/tmp/Non-existing-file.txt" ] ; then | ||
echo "I found the non-existing file" | echo "I found the non-existing file" | ||
| Line 107: | Line 107: | ||
: # the colon command prevents an error if there are no other statements in this block | : # the colon command prevents an error if there are no other statements in this block | ||
fi | fi | ||
</ | </syntaxhighlight> | ||
== Using Find == | == Using Find == | ||
| Line 115: | Line 115: | ||
=== Mime report === | === Mime report === | ||
This example finds and counts files by their extension. A "poor-man's mime-report" | This example finds and counts files by their extension. A "poor-man's mime-report" | ||
< | <syntaxhighlight lang="bash" line> | ||
for x in \ | for x in \ | ||
$(find . -maxdepth 1 -type d | \ | $(find . -maxdepth 1 -type d | \ | ||
| Line 127: | Line 127: | ||
uniq -c ; \ | uniq -c ; \ | ||
done | done | ||
</ | </syntaxhighlight> | ||
Breakdown: | Breakdown: | ||
<pre> | <pre> | ||
| Line 142: | Line 142: | ||
But it looks more impressive as a one-liner: | But it looks more impressive as a one-liner: | ||
< | <syntaxhighlight lang="bash">for x in $(find . -maxdepth 1 -type d|sort|grep -v ^./$); do echo -e "\n\n$x\n"; find "$x" -type f | egrep -o '\.(.?.?..)$' | sort | uniq -c ; done</syntaxhighlight> | ||
=== Prune === | === Prune === | ||
In case you are trying to figure out the prune option so that you can efficiently scan a directory for something while also ignoring .svn metadata, here is an example: | In case you are trying to figure out the prune option so that you can efficiently scan a directory for something while also ignoring .svn metadata, here is an example: | ||
< | <syntaxhighlight lang="bash"> | ||
find ./ -name .svn -prune -o -name "*html*" | find ./ -name .svn -prune -o -name "*html*" | ||
</ | </syntaxhighlight> | ||
Or, a more complex example: 'wcgrep', from the contrib section of the svn repo: | Or, a more complex example: 'wcgrep', from the contrib section of the svn repo: | ||
< | <syntaxhighlight lang="bash"> | ||
#!/bin/bash | #!/bin/bash | ||
| Line 210: | Line 210: | ||
-type f -print0 | xargs -r0 ${WCGREP_GREP:-grep} ${WCGREP_GREPARGS:--HnI} \ | -type f -print0 | xargs -r0 ${WCGREP_GREP:-grep} ${WCGREP_GREPARGS:--HnI} \ | ||
$grepargs "$pattern" | $grepargs "$pattern" | ||
</ | </syntaxhighlight> | ||
== Examples == | == Examples == | ||
This script sets the svn:executable property on a number of files | This script sets the svn:executable property on a number of files | ||
< | <syntaxhighlight lang="bash"> | ||
#!/bin/bash | #!/bin/bash | ||
| Line 239: | Line 239: | ||
echo "Finished fixing websites" | echo "Finished fixing websites" | ||
echo | echo | ||
</ | </syntaxhighlight> | ||
Sometimes when using find, you end up with "Permission denied" errors that add noise to your output. There are a couple solutions for this. Use the '''prune''' option to skip entire trees that you should avoid (e.g. /proc). Use shell redirection to ignore remaining error messages (e.g. 2>/dev/null). | Sometimes when using find, you end up with "Permission denied" errors that add noise to your output. There are a couple solutions for this. Use the '''prune''' option to skip entire trees that you should avoid (e.g. /proc). Use shell redirection to ignore remaining error messages (e.g. 2>/dev/null). | ||
The following example searches all of my hard drive starting at / but skips over the backups directory I have in my external disk drive and also skips over the "process" directory. Any errors like files in /var that I do not have permission to see are discarded by redirecting STDERR to the bitbucket. | The following example searches all of my hard drive starting at / but skips over the backups directory I have in my external disk drive and also skips over the "process" directory. Any errors like files in /var that I do not have permission to see are discarded by redirecting STDERR to the bitbucket. | ||
< | <syntaxhighlight lang="bash"> | ||
find / -path /media/disk/backups -prune -o -path /proc -prune -o -type d -name soffice.cfg 2>/dev/null | find / -path /media/disk/backups -prune -o -path /proc -prune -o -type d -name soffice.cfg 2>/dev/null | ||
/home/greg/.openoffice.org2/user/config/soffice.cfg | /home/greg/.openoffice.org2/user/config/soffice.cfg | ||
| Line 256: | Line 256: | ||
/proc | /proc | ||
/media/disk/backups | /media/disk/backups | ||
</ | </syntaxhighlight> | ||
=== Move a directory up one level === | === Move a directory up one level === | ||
Sometimes you can end up with a directory which is nested inside it's intended destination. For example, <code>drush archive-restore (arr)</code> can leave you with <tt>/var/www/drush/drush</tt> and you want the contents of the sub-directory to be at the location of it's parent. Using the BASH shell options for glob control, you can set dotglob and later unset it to be able to move * up. This worked for me on one host, and didn't work on another. For the one that didn't work, mv kept complaining that destination directories were not empty. I don't care if the destination directories exist.... that's the whole point. Uggh! | Sometimes you can end up with a directory which is nested inside it's intended destination. For example, <code>drush archive-restore (arr)</code> can leave you with <tt>/var/www/drush/drush</tt> and you want the contents of the sub-directory to be at the location of it's parent. Using the BASH shell options for glob control, you can set dotglob and later unset it to be able to move * up. This worked for me on one host, and didn't work on another. For the one that didn't work, mv kept complaining that destination directories were not empty. I don't care if the destination directories exist.... that's the whole point. Uggh! | ||
< | <syntaxhighlight lang="bash"> | ||
cd /var/www/drupal/drupal/ | cd /var/www/drupal/drupal/ | ||
shopt -s dotglob | shopt -s dotglob | ||
mv -- * .. | mv -- * .. | ||
shopt -u dotglob | shopt -u dotglob | ||
</ | </syntaxhighlight> | ||
==Resources== | ==Resources== | ||
| Line 275: | Line 275: | ||
* [http://penguinpetes.com/b2evo/index.php?title=how_the_one_liner_for_loop_in_bash_goes Penguin Pete] | * [http://penguinpetes.com/b2evo/index.php?title=how_the_one_liner_for_loop_in_bash_goes Penguin Pete] | ||
* [[wp:Bash|Wikipedia page]] | * [[wp:Bash|Wikipedia page]] | ||
* [https://ss64.com/bash/test.html Bash tests] <code>-f -d -x -w -Z </code> What do all the file test options mean? | |||
== Bash vs Python or "BASH is better" == | |||
Most job postings that focus on DevOps have requirements for [[Python]], [[Go]] programming or some other programming language. I disagree that a [[DevOps]] Engineer should also be a programmer. I prioritize quality and workmanship (craftsmanship) which is '''informed''' by broad experience, but '''honed''' by specialization. As a construction analogy, I prefer individual skilled trades over the general handyman approach. Simply put: DevOps is DevOps, it is not programming. Worse, the requirement for the hot language of the day is a bigger tell-tale sign that the company is either posturing or doesn't know what they're doing. Back when when I first learned Perl (which isn't the hot new language anymore), there was a hilarious t-shirt that said "Be careful or I'll replace you with a line of code"<ref>Dave Jacoby agrees with me on the broad point that (programming) languages are just different domain dialects, and also cites the ThinkGeek t-shirt phrase "Go Away Or I Will Replace You With a Small Shell Script" | |||
https://jacoby.github.io/2021/11/16/i-will-replace-you-with-a-small-shell-script.html</ref>. Although you ''could'' write the Python example more concisely, it is a real-world example of code that I found that does the same thing as 5 lines of BASH that I wrote. | |||
[[Bash]] code to concatenate [[certbot]] certificates:<syntaxhighlight lang="bash"> | |||
#!/bin/bash | |||
# $RENEWED_DOMAINS will contain a space-delimited list of renewed | |||
# certificate domains (for example, "example.com www.example.com" | |||
# loop through a dynamic list of directories in 'live' | |||
# for SITE in $(find /etc/letsencrypt/live -mindepth 1 -maxdepth 1 -type d -exec basename {} \;) | |||
# $RENEWED_LINEAGE will contain the live subdirectory | |||
for SITE in $RENEWED_DOMAINS | |||
do | |||
# move to correct let's encrypt directory | |||
cd $RENEWED_LINEAGE | |||
# cat files to make combined .pem for haproxy | |||
cat fullchain.pem privkey.pem > /etc/haproxy/certs/$SITE.pem | |||
done | |||
# reload haproxy | |||
# systemctl reload haproxy | |||
</syntaxhighlight>Python code to concatenate certbot certificates:<syntaxhighlight lang="python3"> | |||
#!/usr/bin/env python3 | |||
import os | |||
import re | |||
import sys | |||
# Certbot sets an environment variable RENEWED_LINEAGE, which points to the | |||
# path of the renewed certificate. We use that path to determine and find | |||
# the files for the currently renewed certificated | |||
lineage=os.environ.get('RENEWED_LINEAGE') | |||
# If nothing renewed, exit | |||
if not lineage: | |||
sys.exit() | |||
# From the linage, we strip the 'domain name', which is the last part | |||
# of the path. | |||
result = re.match(r'.*/live/(.+)$', lineage) | |||
# If we can not recognize the path, we exit with 1 | |||
if not result: | |||
sys.exit(1) | |||
# Extract the domain name | |||
domain = result.group(1) | |||
# Define a path for HAproxy where you want to write the .pem file. | |||
deploy_path="/etc/haproxy/ssl/" + domain + ".pem" | |||
# The source files can be found in below paths, constructed with the lineage | |||
# path | |||
source_key = lineage + "/privkey.pem" | |||
source_chain = lineage + "/fullchain.pem" | |||
# HAproxy requires to combine the key and chain in one .pem file | |||
with open(deploy_path, "w") as deploy, \ | |||
open(source_key, "r") as key, \ | |||
open(source_chain, "r") as chain: | |||
deploy.write(key.read()) | |||
deploy.write(chain.read()) | |||
# Here you can add your service reload command. Which will be executed after | |||
# every renewal, which is fine if you only have a few domains. | |||
# Alternative is to add the reload to the --post-hook. In that case it is only | |||
# run once after all renewals. That would be the use-case if you have a large | |||
# number of different certificates served by HAproxy. | |||
</syntaxhighlight> | |||
[[Category:System Administration]] | [[Category:System Administration]] | ||