# The Complete Bash Scripting Tutorial ### From First Script to System Programming, Build Automation & Production-Grade Tools --- ## How to Use This Tutorial This is a comprehensive, professional reference for Bash scripting, organized to be read start‑to‑finish by a beginner, or used as a jump‑to reference by an experienced engineer. Every concept includes runnable code. Tested against **Bash 5.x** on Linux (examples note when something is Bash‑specific vs. POSIX `sh`‑portable). **Companion document:** see `powershell_tutorial.md` for the PowerShell equivalent, and `bash_vs_powershell_and_projects.md` for a side‑by‑side comparison and full capstone projects written in both languages. --- ## Table of Contents **Part I — Foundations** 1. Introduction & Environment Setup 2. Script Anatomy, Execution & Permissions 3. Variables, Data Types & Quoting **Part II — Control Flow** 4. Conditionals: `if`, `test`, `[[ ]]`, `case` 5. Loops: `for`, `while`, `until`, `select` 6. Functions **Part III — Data Structures** 7. Arrays & Associative Arrays 8. String Manipulation & Parameter Expansion 9. Arithmetic Operations **Part IV — I/O & Process Control** 10. Redirection, Pipes & File Descriptors 11. Process Management, Jobs, Signals & Traps 12. Subshells, Command Substitution & Process Substitution **Part V — Text Processing Mastery** 13. `grep`, `sed`, `awk` & the Core Utilities Toolbox **Part VI — System Programming** 14. Reading the System: `/proc`, Permissions, Ownership, Networking 15. Scheduling & Services: `cron`, `at`, `systemd` **Part VII — Configuration & Build Automation** 16. Configuration Files: Parsing, Generating, Templating 17. Build Automation & Compilers: `gcc`/`clang`, `make`, `CMake`, CI **Part VIII — Professional Engineering** 18. Argument Parsing & CLI Design 19. Error Handling, Debugging & Defensive Scripting 20. Security & Performance Best Practices **Part IX — Real‑World Projects** 21. Four Complete Projects (Health Check, Backup, Log Analyzer, Deploy Script) **Appendix** A. Bash Cheat Sheet B. Further Resources --- # Part I — Foundations ## Chapter 1 — Introduction & Environment Setup ### What Bash Is, and Why It Matters Bash ("Bourne Again SHell") is the default command interpreter on almost every Linux distribution and macOS (until Catalina switched the default to `zsh`, though Bash remains universally available), and ships inside Windows via WSL and Git Bash. It is simultaneously: - An **interactive shell** — what you type at a terminal prompt. - A **scripting language** — a full language with variables, functions, loops, and data structures, designed to *orchestrate other programs*. This second point is the key mental model for the entire tutorial: **Bash is glue.** Its native strengths are string handling, file manipulation, and gluing together small, sharp Unix tools (`grep`, `sed`, `awk`, `find`, `curl`...) into pipelines. It is not optimized for heavy computation, complex data structures, or large codebases — for those, reach for Python, Go, or Rust and call them from Bash instead. ### Checking Your Environment ```bash # Which shell am I in right now? echo $SHELL # the shell registered for your user (login shell) ps -p $$ # the shell actually running this process bash --version # Bash version — features differ a lot between 3.x and 5.x # Is a script being run with bash or sh (dash)? Critical for portability. echo $0 ``` macOS still ships Bash 3.2 (licensing reasons) — install a modern version via Homebrew (`brew install bash`) if you need associative arrays, `mapfile`, or other Bash‑4+ features. ### Setting Up a Productive Workflow | Tool | Purpose | |---|---| | `shellcheck` | Static analysis — catches quoting bugs, unsafe patterns. **Install this. Use it on every script.** | | `shfmt` | Consistent formatting | | `bash -n script.sh` | Syntax check without executing | | `bash -x script.sh` | Trace execution (see Ch. 19) | | VS Code + "Bash IDE" extension | Linting, hover docs, go‑to‑definition | ```bash # Install shellcheck (Debian/Ubuntu) sudo apt install shellcheck -y shellcheck myscript.sh ``` Make `shellcheck` part of your habit from day one — most "Bash is unreliable" complaints are actually unquoted-variable bugs that `shellcheck` catches instantly. --- ## Chapter 2 — Script Anatomy, Execution & Permissions ### The Shebang The first line of a script tells the OS which interpreter to use: ```bash #!/usr/bin/env bash # ^ resolves "bash" via $PATH — portable across systems where bash # isn't always at /bin/bash (e.g., some BSDs, Nix, macOS+Homebrew) ``` Prefer `#!/usr/bin/env bash` over the hardcoded `#!/bin/bash` unless you have a specific reason to pin the path (e.g., security‑sensitive system scripts where you don't want `$PATH` consulted). ### Making a Script Executable ```bash cat > hello.sh << 'EOF' #!/usr/bin/env bash echo "Hello, $(whoami). Today is $(date '+%Y-%m-%d')." EOF chmod +x hello.sh # add execute permission ./hello.sh # run it directly bash hello.sh # or run it explicitly without +x ``` ### Permission Bits, Refresher ```bash chmod 755 script.sh # rwxr-xr-x — owner: all, group/others: read+execute chmod u+x script.sh # add execute for owner only chmod go-w script.sh # remove write for group/others ``` `755` (or `750` if group‑shared but not world‑readable) is the standard for scripts you intend to run as executables. ### Exit Codes — The Most Important Convention in Shell Scripting Every command returns an **exit status**: `0` means success, `1–255` means some kind of failure (the specific number is command‑defined). ```bash grep "pattern" file.txt echo $? # 0 if found, 1 if not found, 2 if file.txt doesn't exist # Explicit exit codes in your own scripts exit 0 # success exit 1 # generic failure exit 64 # EX_USAGE, by BSD sysexits.h convention — good practice for CLI tools ``` Your scripts should **always** set a meaningful exit code — calling code (cron, CI pipelines, other scripts) depends on it. ### Comments & Documentation Headers ```bash #!/usr/bin/env bash ############################################################################### # Name: deploy.sh # Description: Deploys the current build artifact to the target environment. # Usage: ./deploy.sh [--dry-run] # Author: Your Name # Requires: bash >= 4.4, jq, rsync ############################################################################### ``` A short header block like this is a professional norm — it tells the next engineer (often future‑you) what a script does without reading the whole thing. --- ## Chapter 3 — Variables, Data Types & Quoting ### Variables Are Untyped Strings (Mostly) Bash has no real type system — everything is a string unless you tell it otherwise (arrays, and integers via `declare -i`/arithmetic context). ```bash name="Hawaii" # NO spaces around '=' — this is a syntax error: name = "Hawaii" age=30 echo "$name is $age" # Variables are visible only in the current shell unless exported export PATH="$PATH:/usr/local/bin" # now visible to child processes too ``` ### `declare` — Explicit Typing & Attributes ```bash declare -i count=5 # integer — arithmetic happens automatically count+=1 # 6, not "51" declare -r PI=3.14159 # readonly — reassignment errors out declare -x API_KEY="abc" # equivalent to export declare -a my_array # indexed array (Chapter 7) declare -A my_map # associative array (Chapter 7) declare -l lower="HELLO" # forces lowercase: "hello" declare -u upper="hello" # forces uppercase: "HELLO" ``` ### Variable Scope ```bash g_var="global" demo() { local l_var="local to demo()" g_var="modified" echo "$l_var" } demo echo "$g_var" # "modified" — globals are visible/mutable everywhere echo "$l_var" # empty — local never escaped the function ``` **Always use `local` for variables inside functions** unless you specifically intend to mutate global state. Forgetting this is one of the most common sources of subtle bugs in larger scripts. ### Quoting — The Single Most Important Bash Skill | Style | Behavior | |---|---| | `'single quotes'` | Literal. No expansion of `$vars`, `` `cmds` ``, or `\`. | | `"double quotes"` | Expands `$vars`, `` `cmds` ``/`$(...)`, but suppresses globbing & word‑splitting. | | no quotes | Expansion **and** word‑splitting **and** globbing happen — almost always wrong for variables. | ```bash file="my report.txt" ls $file # BUG: expands to `ls my report.txt` → looks for two files ls "$file" # correct: `ls "my report.txt"` → one argument names='*' echo $names # BUG: globs — expands to every file in cwd echo "$names" # correct: literally prints * ``` **Rule of thumb: quote every variable expansion (`"$var"`) unless you have a specific, understood reason not to.** This single habit eliminates the majority of real‑world Bash bugs (word‑splitting on filenames with spaces, unintended globbing, etc.). ### Default Values & Parameter Expansion Basics ```bash echo "${name:-Guest}" # use "Guest" if name is unset OR empty (doesn't change $name) echo "${name:=Guest}" # same, but ALSO assigns Guest to $name if unset/empty echo "${name:?Error: name required}" # print error & exit if unset/empty echo "${name:+set}" # print "set" only if name IS set (else empty) ``` (Full parameter‑expansion reference — substrings, case conversion, search/replace — is in Chapter 8.) ### Command‑Line Arguments & Special Variables ```bash #!/usr/bin/env bash echo "Script name: $0" echo "First arg: $1" echo "Second arg: $2" echo "All args (separate words): $@" echo "All args (one string): $*" echo "Arg count: $#" echo "Last exit code: $?" echo "Current script PID: $$" echo "Last background PID: $!" ``` ```bash ./script.sh alpha beta gamma # $0=./script.sh $1=alpha $2=beta $3=gamma $#=3 ``` `"$@"` (quoted, with the `$`) is almost always what you want when forwarding arguments — it preserves each argument as a separate word even if it contains spaces. `$*` joins everything into a single string and is rarely correct. --- # Part II — Control Flow ## Chapter 4 — Conditionals: `if`, `test`, `[[ ]]`, `case` ### `if` / `elif` / `else` ```bash #!/usr/bin/env bash read -rp "Enter a number: " n if (( n > 100 )); then echo "Big" elif (( n > 10 )); then echo "Medium" else echo "Small" fi ``` ### `test`, `[ ]`, and `[[ ]]` — Know the Differences `[ ... ]` is the `test` command (a real external/builtin command — word‑splitting and globbing rules apply to its arguments). `[[ ... ]]` is a **Bash keyword** with safer parsing, pattern matching, and regex support. **Prefer `[[ ]]` in Bash scripts; reserve `[ ]` only for POSIX `sh` portability.** ```bash # [[ ]] - Bash-only, safer [[ -z "$var" ]] # true if var is empty/unset [[ -n "$var" ]] # true if var is non-empty [[ "$a" == "$b" ]] # string equality — no need to quote to avoid glob issues [[ "$file" == *.txt ]] # pattern matching (glob-style) — only inside [[ ]] [[ "$str" =~ ^[0-9]+$ ]] # regex matching, result in $BASH_REMATCH # Logical combinators inside [[ ]] [[ -f "$file" && -r "$file" ]] && echo "exists and readable" [[ "$x" == "a" || "$x" == "b" ]] ``` ### File Test Operators (work in both `[ ]` and `[[ ]]`) | Test | Meaning | |---|---| | `-e file` | exists | | `-f file` | regular file | | `-d file` | directory | | `-L file` | symbolic link | | `-r/-w/-x file` | readable/writable/executable | | `-s file` | exists and size > 0 | | `f1 -nt f2` | f1 newer than f2 | | `f1 -ot f2` | f1 older than f2 | ### Numeric vs String Comparison — A Classic Pitfall ```bash [[ "$a" -eq "$b" ]] # NUMERIC equality — use -eq -ne -lt -le -gt -ge [[ "$a" == "$b" ]] # STRING equality — use == != < > (note: < > need [[ ]] AND escaping in [ ]) (( a == b )) # arithmetic context — most natural for numbers (Chapter 9) ``` `"10" -gt "9"` is `true` (numeric); `"10" > "9"` as strings is `false` (lexical comparison — "1" < "9"). Mixing these up silently produces wrong results — pick the comparison style that matches your data. ### `case` — Pattern‑Matching Switch ```bash read -rp "y/n? " answer case "$answer" in y|Y|yes|YES) echo "Confirmed" ;; n|N|no|NO) echo "Declined" ;; "") echo "No input" ;; *) echo "Unrecognized: $answer" ;; esac ``` `case` supports glob patterns (`*.txt`, `[0-9]*`), making it excellent for dispatching on file extensions, command‑line subcommands, or OS detection: ```bash case "$(uname -s)" in Linux*) echo "Linux" ;; Darwin*) echo "macOS" ;; CYGWIN*|MINGW*) echo "Windows (POSIX layer)" ;; *) echo "Unknown OS" ;; esac ``` --- ## Chapter 5 — Loops: `for`, `while`, `until`, `select` ### `for` — Iterating Lists ```bash # Over an explicit list for color in red green blue; do echo "Color: $color" done # Over a glob (files) for file in /var/log/*.log; do [[ -f "$file" ]] || continue # guard against no-match literal glob echo "Processing $file" done # C-style numeric for for ((i = 0; i < 5; i++)); do echo "i=$i" done # Over a range for n in {1..10}; do echo "$n"; done for n in {0..20..5}; do echo "$n"; done # step of 5 ``` ### `while` and `until` ```bash count=0 while (( count < 5 )); do echo "count=$count" ((count++)) done # Reading a file line-by-line — THE correct idiom while IFS= read -r line; do echo "Line: $line" done < "input.txt" # IFS= preserves leading/trailing whitespace; -r prevents backslash interpretation. # This is markedly safer than `for line in $(cat file)` which word-splits on whitespace. until (( count == 0 )); do echo "Counting down: $count" ((count--)) done ``` ### Reading Command Output Line-by-Line (Without a Subshell Trap) ```bash # BUG-PRONE: the while loop runs in a subshell, so variables set inside it # (e.g., a counter) are LOST after the pipe ends: count=0 ps aux | while read -r line; do ((count++)); done echo "$count" # prints 0 -- not what you expect! # FIX: process substitution avoids the subshell (Chapter 12) count=0 while read -r line; do ((count++)); done < <(ps aux) echo "$count" # correct count ``` ### `select` — Quick Interactive Menus ```bash echo "Choose an environment:" select env in "staging" "production" "quit"; do case "$env" in staging) echo "Deploying to staging"; break ;; production) echo "Deploying to production"; break ;; quit) echo "Bye"; exit 0 ;; *) echo "Invalid choice ($REPLY)" ;; esac done ``` ### `break` and `continue` ```bash for i in {1..10}; do (( i == 5 )) && continue # skip 5 (( i == 8 )) && break # stop at 8 echo "$i" done # With nested loops, break/continue N levels out: for i in {1..3}; do for j in {1..3}; do (( j == 2 )) && continue 2 # continue the OUTER loop echo "$i,$j" done done ``` --- ## Chapter 6 — Functions ### Definition & Calling ```bash # Two equivalent syntaxes — the first is POSIX-portable greet() { echo "Hello, $1!" } function greet2() { # Bash-only syntax, identical behavior echo "Hi, $1!" } greet "Hawaii" # functions are called like commands — no parentheses ``` ### Parameters, Return Values & Output Functions don't "return" data the way other languages do — `return` only sets an **exit code (0–255)**. To return *data*, `echo` it and capture with command substitution, or use a `nameref`/global variable for complex data. ```bash is_even() { local n="$1" (( n % 2 == 0 )) # arithmetic truth value becomes the function's exit code } if is_even 4; then echo "even"; fi add() { local sum=$(( $1 + $2 )) echo "$sum" # "returning" by printing to stdout } result=$(add 3 4) # capture via command substitution echo "Result: $result" ``` ### Passing Arrays & Returning Complex Data (Bash 4.3+) ```bash fill_array() { local -n ref=$1 # nameref: ref becomes an alias for the caller's variable ref=(one two three) } declare -a my_arr fill_array my_arr echo "${my_arr[@]}" # one two three ``` ### Local Variables, Recursion & `$FUNCNAME` ```bash factorial() { local n="$1" if (( n <= 1 )); then echo 1 else local sub sub=$(factorial $(( n - 1 ))) echo $(( n * sub )) fi } echo "$(factorial 5)" # 120 # Stack of currently-executing function names trace() { echo "Call stack: ${FUNCNAME[*]}" } ``` ### Default Parameters & Variadic Arguments ```bash greet() { local name="${1:-World}" # default if not supplied shift || true # drop $1; ignore error if there was no $1 echo "Hello, $name!" echo "Extra args: $*" } greet greet "Hawaii" extra1 extra2 ``` ### Exporting Functions to Subshells/Scripts ```bash my_func() { echo "I'm exported"; } export -f my_func bash -c 'my_func' # works — child bash process inherited the function ``` --- # Part III — Data Structures ## Chapter 7 — Arrays & Associative Arrays ### Indexed Arrays ```bash fruits=("apple" "banana" "cherry") fruits+=("date") # append fruits[10]="mango" # sparse — indices need not be contiguous echo "${fruits[0]}" # apple echo "${fruits[@]}" # all elements, each a separate word echo "${fruits[*]}" # all elements as one string (joined by IFS) echo "${#fruits[@]}" # element count echo "${!fruits[@]}" # all INDICES (useful for sparse arrays) unset 'fruits[1]' # remove banana (leaves a hole) for f in "${fruits[@]}"; do echo "$f"; done # always quote "${arr[@]}"! ``` ### Slicing & Bulk Operations ```bash nums=(10 20 30 40 50) echo "${nums[@]:1:3}" # 20 30 40 (offset 1, length 3) echo "${nums[@]: -2}" # 40 50 (last two — note the required space before -2) # Reading a command's output directly into an array mapfile -t lines < input.txt # one array element per line readarray -t users < <(cut -d: -f1 /etc/passwd) ``` ### Associative Arrays (Bash 4+) ```bash declare -A config config[host]="localhost" config[port]="8080" config["db name"]="mydb" # keys can contain spaces if quoted echo "${config[host]}" echo "${!config[@]}" # all keys echo "${config[@]}" # all values for key in "${!config[@]}"; do echo "$key => ${config[$key]}" done [[ -v config[host] ]] && echo "host is set" # check key existence (Bash 4.2+) ``` Associative arrays are the natural structure for representing config data, lookup tables, or counting occurrences (`counts["$word"]=$(( ${counts["$word"]:-0} + 1 ))`). --- ## Chapter 8 — String Manipulation & Parameter Expansion Bash's **parameter expansion** syntax (`${var...}`) is dense but covers most string operations natively — no external `sed`/`awk` call required for simple cases, which matters for performance in hot loops. ### Length, Substrings, Case ```bash s="Hello, World!" echo "${#s}" # 13 — length echo "${s:7}" # "World!" — substring from index 7 echo "${s:7:5}" # "World" — substring, offset 7, length 5 echo "${s: -6}" # "World!" — from the end (space before - required) echo "${s^^}" # HELLO, WORLD! — uppercase all echo "${s,,}" # hello, world! — lowercase all echo "${s^}" # Hello, World! — uppercase first char only ``` ### Search & Replace ```bash path="/usr/local/bin/script.sh" echo "${path/bin/sbin}" # replace FIRST match: /usr/local/sbin/script.sh echo "${path//\//_}" # replace ALL matches: _usr_local_bin_script.sh echo "${path#*/}" # remove SHORTEST match from front: usr/local/bin/script.sh echo "${path##*/}" # remove LONGEST match from front: script.sh (= basename!) echo "${path%/*}" # remove SHORTEST match from back: /usr/local/bin (= dirname!) echo "${path%%.*}" # remove LONGEST match from back: /usr/local/bin/script ``` The `#`/`##`/`%`/`%%` family is how experienced Bash users implement `basename`/`dirname`/extension‑stripping without spawning external processes — meaningfully faster in loops over many files. ### Default/Alternate Values (recap with more detail) ```bash unset var echo "${var-default}" # "default" (unset only — empty string would print as empty) echo "${var:-default}" # "default" (unset OR empty) var="" echo "${var-default}" # "" (var IS set, just empty) echo "${var:-default}" # "default" (treats empty as unset too) ``` ### Splitting Strings into Arrays ```bash csv="alpha,beta,gamma" IFS=',' read -ra parts <<< "$csv" echo "${parts[1]}" # beta # Or with parameter expansion + IFS manipulation: IFS=',' parts=($csv) # legacy style — works but unquoted; prefer read -ra above ``` ### Joining an Array into a String ```bash arr=(a b c) joined=$(IFS=,; echo "${arr[*]}") echo "$joined" # a,b,c # Bash 4.4+: printf trick for arbitrary delimiters printf -v joined '%s,' "${arr[@]}" joined="${joined%,}" # strip trailing delimiter ``` --- ## Chapter 9 — Arithmetic Operations ### Integer Arithmetic — `(( ))` and `$(( ))` ```bash a=5; b=3 echo $(( a + b )) # 8 echo $(( a ** b )) # 125 — exponentiation echo $(( a % b )) # 2 — modulo (( a > b )) && echo "a is bigger" (( count++ )) # increment — no $ needed inside (( )) (( total += 10 )) # Bases echo $(( 0x1F )) # 31 — hex literal echo $(( 0755 )) # 493 — octal literal echo $(( 2#1010 )) # 10 — base#value notation, base 2 ``` Bash arithmetic is **integer only**. `$(( 5 / 2 ))` is `2`, not `2.5`. ### Floating Point — Delegate to `bc` or `awk` ```bash echo "scale=4; 22/7" | bc -l # 3.1428 result=$(awk "BEGIN { print 22/7 }") # 3.14286 — often more convenient in scripts python3 -c "print(22/7)" # also perfectly idiomatic if Python is available ``` ### `let` and `expr` (Legacy, Generally Avoid) ```bash let "x = 5 + 3" # works but $(( )) is clearer and safer (no word-splitting risk) y=$(expr 5 + 3) # ancient, slow (external process), heavily superseded by $(( )) ``` `expr` predates `$(( ))` and `[[ ]]` and should be treated as legacy — only reach for it in scripts that must run under a strict POSIX `sh` with no arithmetic expansion at all. --- # Part IV — I/O & Process Control ## Chapter 10 — Redirection, Pipes & File Descriptors ### The Three Standard Streams Every process starts with three open file descriptors: `0` (stdin), `1` (stdout), `2` (stderr). ```bash command > output.txt # redirect stdout, OVERWRITE command >> output.txt # redirect stdout, APPEND command 2> errors.txt # redirect stderr only command > out.txt 2>&1 # stdout AND stderr to the same file (order matters! see below) command &> all.txt # Bash shorthand for the line above command < input.txt # redirect stdin from a file command < /dev/null # ensure no stdin is read (common in cron/daemons) command > /dev/null 2>&1 # discard all output entirely ``` **Order matters with `2>&1`:** ```bash command > out.txt 2>&1 # correct: stdout -> file, THEN stderr duplicated to wherever stdout NOW points (the file) command 2>&1 > out.txt # WRONG for combining: stderr duplicated to the OLD stdout (terminal) first, THEN stdout redirected to file — stderr still goes to terminal ``` ### Pipes ```bash ps aux | grep nginx | awk '{print $2}' cat access.log | sort | uniq -c | sort -rn | head -10 # top 10 most frequent lines ``` `PIPESTATUS` gives you the exit code of *each* stage in the last pipeline (the overall `$?` only reflects the last command): ```bash false | true echo "${PIPESTATUS[@]}" # 1 0 — see that the `false` failed even though overall $? is 0 set -o pipefail # makes the WHOLE pipeline fail if ANY stage fails (Chapter 19) ``` ### Here-Documents & Here-Strings ```bash cat << INNERDOC Multi-line text. Variables ARE expanded: $HOME INNERDOC cat << 'INNERDOC' Variables are NOT expanded here because the delimiter is quoted: $HOME stays literal. INNERDOC cat <<- INNERDOC This here-doc strips LEADING TABS (note the dash after <<), useful for keeping indentation clean in nested code without polluting output. INNERDOC grep "pattern" <<< "single line here-string, no need for echo | grep" ``` ### Custom File Descriptors ```bash exec 3> log.txt # open fd 3 for writing to log.txt echo "logged line" >&3 # write to it exec 3>&- # close fd 3 # Read and write a file simultaneously without clobbering (in-place-ish editing) exec 3<> data.txt read -r line <&3 echo "Read: $line" exec 3>&- ``` ### `tee` — Duplicate a Stream ```bash echo "hello" | tee output.txt # writes to file AND prints to terminal build_command | tee -a build.log # append while still showing live output some_cmd 2>&1 | tee debug.log # capture stderr+stdout while watching live ``` --- ## Chapter 11 — Process Management, Jobs, Signals & Traps ### Foreground, Background & Job Control ```bash long_task.sh & # run in background; shell prints a job number and PID jobs # list background jobs in this shell fg %1 # bring job 1 to foreground bg %1 # resume job 1 in background kill %1 # send SIGTERM to job 1 wait # block until ALL background jobs finish wait $! # block until the most recently backgrounded PID finishes wait -n # wait for the NEXT job to finish (Bash 4.3+) ``` ### Running Things in Parallel ```bash for url in "${urls[@]}"; do curl -sO "$url" & # fire off downloads in parallel done wait # wait for them all to complete # Capture each background job's exit code pids=() for cmd in "${commands[@]}"; do eval "$cmd" & pids+=("$!") done fail=0 for pid in "${pids[@]}"; do wait "$pid" || fail=1 done exit "$fail" ``` For real parallelism with throttling (limit to N concurrent jobs), `xargs -P` or GNU `parallel` are far more robust than hand-rolled job arrays — see Chapter 13. ### Signals | Signal | Number | Typical Meaning | |---|---|---| | `SIGHUP` | 1 | Terminal closed / reload config | | `SIGINT` | 2 | Ctrl-C | | `SIGKILL` | 9 | Force kill — cannot be trapped or ignored | | `SIGTERM` | 15 | Polite "please stop" request — the default for `kill` | | `SIGSTOP`/`SIGCONT` | 19/18 | Pause / resume | ```bash kill -15 1234 # polite request to PID 1234 (default) kill -9 1234 # forceful — last resort, no cleanup possible killall nginx # kill by process name instead of PID pkill -f "myscript.sh" # kill by matching full command line ``` ### `trap` — Cleanup & Signal Handling `trap` is essential for **writing scripts that clean up after themselves** — temp files, lock files, background processes — even when interrupted. ```bash #!/usr/bin/env bash tmpfile=$(mktemp) cleanup() { echo "Cleaning up..." rm -f "$tmpfile" } trap cleanup EXIT # runs on ANY exit: normal, error, or signal trap 'echo "Interrupted"; exit 130' INT # Ctrl-C handling specifically echo "Working with $tmpfile..." sleep 30 # try pressing Ctrl-C during this — cleanup still runs ``` `trap ... EXIT` is the closest Bash equivalent to a `finally` block and should be a standard part of any script that creates temp files, acquires locks, or starts background processes. ### Lock Files (Preventing Concurrent Runs) ```bash LOCKFILE="/var/run/myscript.lock" exec 200> "$LOCKFILE" if ! flock -n 200; then echo "Another instance is already running. Exiting." >&2 exit 1 fi # ... critical section ... # lock auto-releases when fd 200 closes (script exit) ``` --- ## Chapter 12 — Subshells, Command Substitution & Process Substitution ### Subshells `( )` ```bash ( cd /tmp && rm -f scratch.txt ) # cd only affects this subshell — cwd unchanged afterward echo "$PWD" # unchanged — proves the subshell didn't leak state ( export DEBUG=1 run_diagnostics.sh ) echo "$DEBUG" # unset — exports inside a subshell don't escape it ``` Subshells are useful for **isolating side effects** (directory changes, variable exports, `set` options) to a specific block of code. ### Command Substitution ```bash now=$(date +%T) # PREFERRED modern syntax now=`date +%T` # legacy backtick syntax — avoid; doesn't nest cleanly nested=$(echo "outer $(echo "inner")") # $(...) nests trivially; backticks require escaping ``` ### Process Substitution — Treat a Command's Output as a "File" ```bash diff <(sort file1.txt) <(sort file2.txt) # compare sorted versions without writing temp files while read -r line; do echo "Got: $line" done < <(grep ERROR app.log) # avoids the subshell-variable-loss problem from Ch.5 tee >(gzip > log.gz) >(wc -l > count.txt) < access.log > /dev/null # fan-out to multiple consumers ``` `<(...)` and `>(...)` are genuinely one of Bash's most powerful, under-used features — they let you plug a command's input/output stream in anywhere a filename is expected. --- # Part V — Text Processing Mastery ## Chapter 13 — `grep`, `sed`, `awk` & the Core Utilities Toolbox This chapter is the heart of practical Bash work: chaining small tools into pipelines that transform text faster than you could write equivalent code in most general-purpose languages. ### `grep` — Searching Text ```bash grep "error" app.log # lines containing "error" grep -i "error" app.log # case-insensitive grep -v "debug" app.log # INVERT — lines NOT containing "debug" grep -c "error" app.log # COUNT matching lines grep -n "error" app.log # show line NUMBERS grep -r "TODO" ./src # RECURSIVE through a directory grep -l "import requests" ./*.py # just FILENAMES that match grep -E "[0-9]{3}-[0-9]{4}" contacts.txt # extended regex (egrep) grep -A 3 -B 1 "Exception" app.log # 3 lines After, 1 Before each match grep -w "cat" file.txt # match WHOLE WORD only (not "category") grep -o -E "[0-9]+\.[0-9]+\.[0-9]+" file.txt # print ONLY the matched text (e.g. version numbers) grep -P '(?<=user=)\w+' file.txt # Perl-compatible regex with lookaround (GNU grep) ``` ### `sed` — Stream Editor ```bash sed 's/foo/bar/' file.txt # replace FIRST occurrence per line sed 's/foo/bar/g' file.txt # replace ALL occurrences per line sed -i 's/foo/bar/g' file.txt # edit the FILE IN PLACE sed -i.bak 's/foo/bar/g' file.txt # in-place, but keep a .bak backup first (safer!) sed -n '5,10p' file.txt # PRINT only lines 5-10 sed '/^#/d' file.txt # DELETE lines starting with # (comments) sed -n '/START/,/END/p' file.txt # print everything BETWEEN two markers sed 's/\(.*\),\(.*\)/\2,\1/' file.csv # swap two comma-separated fields using capture groups sed 's/[[:space:]]*$//' file.txt # strip TRAILING whitespace echo "hello world" | sed 's/\(.\)/\U\1/' # uppercase first letter via case conversion ``` `-i` (in‑place) **has no backup by default on GNU sed** — always test the pattern without `-i` first, or use `-i.bak`, before running it against anything you care about. ### `awk` — Field‑Based Text Processing & Mini‑Language `awk` is effectively a complete language built around the idea of "for each line, split into fields, then act." ```bash awk '{print $1, $3}' data.txt # print 1st and 3rd whitespace-delimited fields awk -F, '{print $2}' data.csv # use comma as the field separator awk '{sum += $2} END {print sum}' sales.txt # accumulate a running total awk '$3 > 100 {print $1}' data.txt # CONDITIONAL: filter rows by field value awk 'NR==1 {print; next} {print | "sort"}' data.txt # keep header, sort the rest awk 'BEGIN {print "Report:"} {print $0} END {print "Done"}' f # BEGIN/END blocks run once each awk '{print NR, NF, $0}' file.txt # NR=line number, NF=field count # A small real report: average response time per endpoint from a log awk -F'"' '{print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -rn | head # Multi-line awk script for clarity on bigger jobs awk ' BEGIN { FS=","; print "Processing CSV..." } { total[$1] += $2 } END { for (key in total) print key, total[key] } ' sales.csv ``` ### Core Utilities Quick Reference ```bash cut -d',' -f1,3 data.csv # extract columns 1 and 3 from CSV sort file.txt # alphabetical sort sort -n file.txt # NUMERIC sort sort -k2,2 -t',' file.csv # sort by the 2nd comma-delimited field sort -u file.txt # sort AND deduplicate uniq -c file.txt # collapse adjacent duplicates, prefix with count (sort first!) tr 'a-z' 'A-Z' < file.txt # translate lowercase to uppercase tr -d '\r' < windows_file.txt # strip carriage returns (CRLF -> LF) tr -s ' ' # squeeze repeated spaces into one wc -l file.txt # line count wc -w file.txt # word count find . -name "*.log" -mtime +7 -delete # delete .log files older than 7 days find . -type f -size +100M # files bigger than 100MB find . -name "*.tmp" -exec rm {} \; # exec a command per result find . -name "*.py" -print0 | xargs -0 grep -l "TODO" # null-delimited — SAFE with weird filenames xargs -P4 -I{} curl -sO {} < urls.txt # 4-way parallel downloads column -t -s, data.csv # pretty-print CSV as an aligned table paste -d, file1.txt file2.txt # merge files line-by-line, comma-joined comm -23 sorted1.txt sorted2.txt # lines unique to sorted1.txt (set difference) ``` ### Why `find ... -print0 | xargs -0` Instead of Plain Piping Filenames can legally contain spaces, newlines, and other special characters. `-print0`/`xargs -0` use the NUL byte as a separator (which can't appear in filenames), making the pipeline correct for *every* possible filename — `find ... | xargs` without `-print0`/`-0` silently breaks on filenames containing spaces. ### Putting It Together: A Realistic One-Liner ```bash # Top 10 IPs hitting your server, from an Nginx-style access log awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10 # Find all large files modified in the last day, sorted by size find /var/log -type f -mtime -1 -printf '%s %p\n' | sort -rn | head -20 ``` --- # Part VI — System Programming ## Chapter 14 — Reading the System: `/proc`, Permissions, Ownership, Networking Bash scripts that inspect and manage the operating system itself — process state, hardware, network sockets — are "system programming" in the Unix sense: most of what you'd do with syscalls in C, Linux exposes as plain text files and CLI tools instead. ### `/proc` — The Kernel's Filesystem Interface ```bash cat /proc/cpuinfo | grep "model name" | head -1 # CPU model cat /proc/meminfo | grep MemAvailable # available RAM cat /proc/loadavg # system load averages cat /proc/uptime # seconds since boot # Inspect a specific process by PID pid=1234 cat /proc/$pid/status | grep -E "^(Name|State|VmRSS)" # name, state, resident memory ls -l /proc/$pid/fd # open file descriptors cat /proc/$pid/cmdline | tr '\0' ' ' # full command line (NUL-separated) readlink /proc/$pid/exe # path to the running binary ``` ### Practical System Info Script ```bash #!/usr/bin/env bash set -euo pipefail echo "=== System Snapshot ===" echo "Hostname: $(hostname)" echo "Kernel: $(uname -r)" echo "Uptime: $(uptime -p)" echo "CPU cores: $(nproc)" echo "Load avg: $(cut -d' ' -f1-3 /proc/loadavg)" mem_total=$(awk '/MemTotal/ {print $2}' /proc/meminfo) mem_avail=$(awk '/MemAvailable/ {print $2}' /proc/meminfo) echo "Memory: $(( (mem_total - mem_avail) / 1024 ))MB used / $(( mem_total / 1024 ))MB total" echo "Disk (/): $(df -h / | awk 'NR==2 {print $3 " used / " $2 " total (" $5 " full)"}')" echo "Top 5 CPU: "; ps -eo pid,comm,%cpu --sort=-%cpu | head -6 | tail -5 ``` ### File Permissions, Ownership & ACLs ```bash chmod 644 file.txt # rw-r--r-- chmod -R 755 ./scripts/ # recursive chown user:group file.txt # change owner & group chown -R www-data:www-data /var/www/html # Symbolic vs numeric — symbolic is often clearer for partial changes chmod u=rwx,g=rx,o= script.sh chmod +t /shared/dir # set the sticky bit (only the owner can delete their own files) chmod g+s /shared/dir # set-group-ID: new files inherit the directory's group umask 022 # default permissions for newly created files (this session) # Access Control Lists for finer-grained permissions than owner/group/other setfacl -m u:alice:rwx /shared/project getfacl /shared/project ``` ### Finding Files by Permission/Ownership (Security Audits) ```bash find / -perm -4000 -type f 2>/dev/null # find SUID binaries (privilege escalation surface) find /home -type f -perm -o+w # world-writable files (typically a misconfiguration) find / -nouser -o -nogroup 2>/dev/null # orphaned files (owner/group no longer exists) ``` ### Networking from the Shell ```bash curl -s -o /dev/null -w "%{http_code}\n" https://example.com # just the HTTP status code curl -sI https://example.com # headers only (HEAD-like) curl -s https://api.example.com/data | jq '.results[0]' # fetch + parse JSON (Ch.16) curl -X POST -H "Content-Type: application/json" -d '{"k":"v"}' https://api.example.com wget -q -O - https://example.com/file.txt # download to stdout nc -zv example.com 443 # check if a TCP port is open ("netcat" port scan/probe) ss -tulnp # list listening sockets (replacement for the deprecated netstat) ip addr show # show network interfaces & IPs dig +short example.com # quick DNS lookup host example.com # alternative DNS lookup # A minimal "wait for service to be ready" loop — common in deploy/CI scripts until curl -sf http://localhost:8080/health > /dev/null; do echo "Waiting for service..." sleep 2 done echo "Service is up." ``` --- ## Chapter 15 — Scheduling & Services: `cron`, `at`, `systemd` ### `cron` — Recurring Jobs ```bash crontab -e # edit your personal crontab crontab -l # list it # Crontab line format: minute hour day-of-month month day-of-week command # * * * * * command 0 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1 # every day at 2:00 AM */15 * * * * /opt/scripts/healthcheck.sh # every 15 minutes 0 0 1 * * /opt/scripts/monthly-report.sh # midnight on the 1st of each month 0 9 * * 1-5 /opt/scripts/weekday-digest.sh # 9 AM, Mon–Fri only ``` **Cron pitfalls (the source of "it works on my terminal but not in cron"):** cron runs with a minimal environment — no full `$PATH`, no interactive shell config loaded. Always use absolute paths in cron scripts (or explicitly `source` the needed environment), and always redirect output to a log file so failures are visible. ### `at` — One‑Time Scheduled Jobs ```bash echo "/opt/scripts/cleanup.sh" | at 23:00 # run once, tonight at 11 PM at now + 30 minutes <<< "/opt/scripts/reminder.sh" atq # list pending at jobs atrm 3 # cancel job number 3 ``` ### `systemd` — Modern Service Management & Timers For anything more than the simplest recurring task, a `systemd` service + timer is more robust than cron: structured logging via `journalctl`, dependency management, automatic restarts. ```ini # /etc/systemd/system/myapp.service [Unit] Description=My Application After=network.target [Service] Type=simple ExecStart=/opt/myapp/run.sh Restart=on-failure RestartSec=5 User=myappuser Environment=NODE_ENV=production [Install] WantedBy=multi-user.target ``` ```ini # /etc/systemd/system/myapp.timer -- replaces a cron line [Unit] Description=Run myapp daily [Timer] OnCalendar=*-*-* 02:00:00 Persistent=true [Install] WantedBy=timers.target ``` ```bash sudo systemctl daemon-reload # re-read unit files after editing sudo systemctl enable --now myapp.service # enable on boot + start now sudo systemctl status myapp.service sudo systemctl restart myapp.service journalctl -u myapp.service -f # follow live logs for the service journalctl -u myapp.service --since "1 hour ago" ``` Writing the **script itself** (`/opt/myapp/run.sh`) is still ordinary Bash — `systemd` is just a far more capable supervisor than cron for starting it, restarting it on failure, and capturing its logs. --- # Part VII — Configuration & Build Automation ## Chapter 16 — Configuration Files: Parsing, Generating, Templating Real-world scripts almost always need to read settings from somewhere other than hardcoded values. This chapter covers the formats you'll meet most often. ### `.env` Files (Key=Value) ```bash # config/.env DB_HOST=localhost DB_PORT=5432 DB_NAME=myapp # comments and blank lines are fine API_KEY="quoted values work too" ``` ```bash # Loading it safely set -a # automatically export every variable set from here on source config/.env set +a echo "$DB_HOST:$DB_PORT/$DB_NAME" ``` Avoid `eval $(cat .env)` or naive `export $(cat .env | xargs)` — both break on values containing spaces or special characters, and `eval` on untrusted input is a code-injection risk. `source`ing a well-formed `.env` is simpler and safer. ### INI-Style Files ```ini ; config.ini [database] host = localhost port = 5432 [server] workers = 4 ``` ```bash # A small, dependency-free INI reader using awk get_ini_value() { local file="$1" section="$2" key="$3" awk -F '=' -v section="[$section]" -v key="$key" ' $0 == section { in_section=1; next } /^\[/ { in_section=0 } in_section && $1 ~ key { gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2; exit } ' "$file" } host=$(get_ini_value config.ini database host) echo "DB host: $host" ``` ### JSON — Use `jq`, Don't Hand-Parse It JSON's nested structure and quoting rules make it genuinely unsafe to parse with `grep`/`sed` for anything beyond the most trivial case. `jq` is the standard, purpose-built tool. ```bash # config.json: {"database": {"host": "localhost", "port": 5432}, "features": ["a", "b"]} host=$(jq -r '.database.host' config.json) port=$(jq -r '.database.port' config.json) features=$(jq -r '.features[]' config.json) # one per line # Generating JSON safely (handles escaping for you, unlike string concatenation) jq -n --arg host "$host" --argjson port "$port" \ '{database: {host: $host, port: $port}}' > config.json # Modifying a key in-place jq '.database.port = 5433' config.json > tmp.json && mv tmp.json config.json # Querying an API response curl -s https://api.example.com/users | jq -r '.[] | "\(.name): \(.email)"' ``` ### YAML — `yq` (the `jq` for YAML) ```bash yq '.database.host' config.yaml yq -i '.database.port = 5433' config.yaml # in-place update yq -o=json '.' config.yaml # convert YAML to JSON ``` ### CSV — `cut`/`awk` for Simple Cases, `csvkit`/`miller` for Real Work ```bash # Simple: extract a column cut -d, -f2 data.csv # Real-world: column with embedded commas/quotes — use a proper CSV-aware tool csvcut -c name,email data.csv mlr --csv filter '$age > 30' data.csv ``` ### Templating Config Files (Generating Output from a Template) ```bash # template.conf.tpl server_name=__HOSTNAME__ listen_port=__PORT__ # Simple sed-based substitution sed -e "s/__HOSTNAME__/$(hostname)/" -e "s/__PORT__/8080/" template.conf.tpl > server.conf # envsubst — purpose-built for ${VAR}-style substitution, handles many vars cleanly export HOSTNAME=$(hostname) PORT=8080 envsubst < template.conf.tpl > server.conf ``` ```bash # Or generate config directly with a heredoc — clean for short files generate_config() { cat << CONF_EOF server_name=$1 listen_port=$2 log_level=${3:-info} CONF_EOF } generate_config "$(hostname)" 8080 > server.conf ``` ### Reading Your Own Script's Configuration with a Function ```bash load_config() { local config_file="${1:-./config.env}" [[ -f "$config_file" ]] || { echo "Config not found: $config_file" >&2; exit 1; } # shellcheck disable=SC1090 source "$config_file" } load_config "$@" ``` --- ## Chapter 17 — Build Automation & Compilers: `gcc`/`clang`, `make`, `CMake`, CI Bash is the glue layer of almost every build system on Linux/macOS — even when the *build* itself is driven by `make`, `CMake`, or a CI YAML file, something underneath is shelling out to compilers via Bash. ### Driving a Compiler Directly ```bash # C — compile a single file gcc -Wall -Wextra -O2 -o myprogram main.c ./myprogram # Multiple source files + a separate headers directory gcc -Wall -I./include -c src/*.c -o build/ # WRONG (-o expects one file for -c); see Makefile below gcc -Wall -I./include -c src/foo.c -o build/foo.o gcc -Wall -I./include -c src/bar.c -o build/bar.o gcc build/foo.o build/bar.o -o build/myprogram # C++ with a modern standard, sanitizers for debugging memory bugs g++ -std=c++20 -Wall -Wextra -fsanitize=address,undefined -g -o app main.cpp # Cross-compiling (targeting ARM from an x86 host) arm-linux-gnueabihf-gcc -o app_arm main.c ``` ### A Hand-Written Build Script (What `make` Actually Automates) ```bash #!/usr/bin/env bash # build.sh — minimal incremental build: only recompile changed .c files set -euo pipefail SRC_DIR="src"; BUILD_DIR="build"; CC="gcc"; CFLAGS="-Wall -Wextra -O2" mkdir -p "$BUILD_DIR" objects=() for src in "$SRC_DIR"/*.c; do obj="$BUILD_DIR/$(basename "${src%.c}.o")" if [[ ! -f "$obj" || "$src" -nt "$obj" ]]; then # only rebuild if source is newer echo "Compiling $src..." $CC $CFLAGS -c "$src" -o "$obj" fi objects+=("$obj") done echo "Linking..." $CC "${objects[@]}" -o "$BUILD_DIR/app" echo "Build complete: $BUILD_DIR/app" ``` This is exactly the dependency logic `make` exists to formalize — once your build has more than a handful of files, switch to a real `Makefile`: ```makefile # Makefile CC := gcc CFLAGS := -Wall -Wextra -O2 -Iinclude SRC := $(wildcard src/*.c) OBJ := $(patsubst src/%.c,build/%.o,$(SRC)) TARGET := build/app $(TARGET): $(OBJ) $(CC) $(OBJ) -o $@ build/%.o: src/%.c @mkdir -p build $(CC) $(CFLAGS) -c $< -o $@ .PHONY: clean clean: rm -rf build ``` ```bash make # builds only what's changed, via the same mtime logic as above, more robustly make clean # remove build artifacts make -j4 # parallel build across 4 jobs ``` ### `CMake` — Generating Build Systems for Larger/Cross-Platform Projects ```cmake # CMakeLists.txt cmake_minimum_required(VERSION 3.16) project(MyApp C) add_executable(myapp src/main.c src/util.c) target_compile_options(myapp PRIVATE -Wall -Wextra) ``` ```bash cmake -S . -B build -DCMAKE_BUILD_TYPE=Release cmake --build build -j$(nproc) ./build/myapp ``` ### Wrapping It All in a CI-Style Script ```bash #!/usr/bin/env bash # ci-build.sh — the kind of script a GitHub Actions/GitLab CI job actually runs set -euo pipefail trap 'echo "Build FAILED at line $LINENO" >&2' ERR echo "==> Installing dependencies" sudo apt-get update -qq && sudo apt-get install -y build-essential cmake echo "==> Configuring" cmake -S . -B build -DCMAKE_BUILD_TYPE=Release echo "==> Building" cmake --build build -j"$(nproc)" echo "==> Running tests" ctest --test-dir build --output-on-failure echo "==> Build artifact" ls -lh build/myapp echo "BUILD SUCCEEDED" ``` This pattern — `set -euo pipefail`, a `trap ... ERR` for diagnostics, clearly labeled `echo "==> Stage"` sections, and a non-zero exit on any real failure — is exactly what's inside the "build step" of most CI systems (GitHub Actions, GitLab CI, Jenkins), even though the surrounding YAML looks like a different system entirely. **Bash is the actual engine; the CI config is just a trigger and an environment.** ### Invoking Other Language Compilers/Toolchains from Bash ```bash # Go go build -o bin/app ./cmd/app && ./bin/app # Rust cargo build --release && ./target/release/app # Java javac -d build src/Main.java && java -cp build Main # TypeScript -> JavaScript npx tsc --outDir dist && node dist/index.js ``` The pattern is always the same: **Bash orchestrates** (install deps, set flags/env vars, run the compiler, run tests, check exit codes, report) while the compiler does the actual heavy lifting. That orchestration role is most of what "build automation" means in practice. --- # Part VIII — Professional Engineering ## Chapter 18 — Argument Parsing & CLI Design ### Positional Arguments (Simple Cases) ```bash src="$1" dst="$2" [[ -z "$src" || -z "$dst" ]] && { echo "Usage: $0 " >&2; exit 64; } ``` ### `getopts` — Standard Short-Option Parsing ```bash #!/usr/bin/env bash usage() { echo "Usage: $0 [-v] [-o output] [-h] "; exit 64; } verbose=0 output="-" while getopts ":vo:h" opt; do case "$opt" in v) verbose=1 ;; o) output="$OPTARG" ;; h) usage ;; \?) echo "Invalid option: -$OPTARG" >&2; usage ;; :) echo "Option -$OPTARG requires an argument" >&2; usage ;; esac done shift $((OPTIND - 1)) # remove parsed options, leaving positional args in $1, $2... input="${1:-}" [[ -z "$input" ]] && usage (( verbose )) && echo "Verbose mode on" echo "Input: $input, Output: $output" ``` `getopts` only supports single-character flags natively (`-v`, `-o value`) — it does **not** support `--long-options` out of the box. ### Manual Parsing for `--long-options` ```bash #!/usr/bin/env bash usage() { echo "Usage: $0 --input FILE [--output FILE] [--verbose] [--help]"; exit 64; } INPUT=""; OUTPUT="-"; VERBOSE=0 while [[ $# -gt 0 ]]; do case "$1" in --input) INPUT="$2"; shift 2 ;; --output) OUTPUT="$2"; shift 2 ;; --verbose) VERBOSE=1; shift ;; --help) usage ;; --*) echo "Unknown option: $1" >&2; usage ;; *) break ;; # first non-option argument — stop parsing flags esac done [[ -z "$INPUT" ]] && { echo "Error: --input is required" >&2; usage; } echo "Input=$INPUT Output=$OUTPUT Verbose=$VERBOSE" ``` ### CLI Design Conventions Worth Following - Support `-h`/`--help` and print usage to **stdout** on success, but to **stderr** with a non-zero exit on actual errors. - Exit code `0` = success, `64` = usage error (BSD `sysexits.h` convention), `1` = generic runtime error — pick a convention and document it. - Read from stdin when no file argument is given (`cat`/`grep`-style tools all do this) — makes your script pipeline-composable. - Print machine-readable output (or `--json`) when stdout isn't a terminal, human-readable when it is — check with `[[ -t 1 ]]`. --- ## Chapter 19 — Error Handling, Debugging & Defensive Scripting ### The Safety Harness: `set -euo pipefail` ```bash #!/usr/bin/env bash set -euo pipefail # -e: exit immediately if any command fails (non-zero exit), instead of plowing on # -u: error on use of an UNSET variable, instead of silently treating it as empty # -o pipefail: a pipeline fails if ANY stage fails, not just the last one IFS=$'\n\t' # optional but common companion — restricts word-splitting to newline/tab only ``` This is widely considered the standard opening for any non-trivial Bash script. It converts entire categories of silent failures into loud, immediate ones — which is exactly what you want during development and in production automation. **Caveats to know:** ```bash set -e grep "pattern" file.txt || true # `|| true` deliberately allows a non-zero exit (e.g. grep "not found") some_command_that_may_fail || { echo "Handled the failure gracefully" >&2 } # set -e does NOT trigger inside a condition (if/while/&&/||) — this is intentional, # not a bug, but it surprises people: if grep "pattern" file.txt; then # grep failing here does NOT exit the script echo "found" fi ``` ### `trap ... ERR` — Centralized Error Reporting ```bash set -eE # -E ensures ERR trap is inherited by functions/subshells too trap 'echo "Error on line $LINENO: command \"$BASH_COMMAND\" exited with $?" >&2' ERR ``` ### Debugging Tools ```bash bash -n script.sh # syntax check ONLY — doesn't execute anything bash -x script.sh # TRACE every command before running it (prefixed with +) set -x # turn tracing on mid-script set +x # turn it back off PS4='+ ${BASH_SOURCE}:${LINENO}: ' # customize trace output to show file:line — extremely useful bash -x script.sh 2> trace.log # capture the trace to a file for later review ``` ```bash # A "debug mode" toggle pattern common in production scripts DEBUG="${DEBUG:-0}" debug_log() { (( DEBUG )) && echo "[DEBUG] $*" >&2; } debug_log "Connecting to $HOST on port $PORT" ``` ### Input Validation — Never Trust External Input ```bash validate_input() { local val="$1" [[ "$val" =~ ^[a-zA-Z0-9_-]+$ ]] || { echo "Invalid input: $val" >&2; return 1; } } # Validate a number is actually a number before doing arithmetic on it [[ "$1" =~ ^[0-9]+$ ]] || { echo "Argument must be a positive integer" >&2; exit 1; } ``` ### Logging That Scales ```bash LOG_FILE="/var/log/myscript.log" log() { local level="$1"; shift printf '%s [%s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$level" "$*" | tee -a "$LOG_FILE" >&2 } log INFO "Starting backup" log ERROR "Failed to connect to $HOST" ``` --- ## Chapter 20 — Security & Performance Best Practices ### Security Checklist ```bash # 1. NEVER eval untrusted input eval "$user_input" # DON'T — arbitrary code execution if $user_input is attacker-controlled # 2. Always quote variables, especially in commands that touch the filesystem rm -rf "$dir"/* # if $dir is empty/unset and unquoted, this can become `rm -rf /*`!! rm -rf "${dir:?}"/* # SAFER: ${var:?} aborts immediately if dir is unset/empty # 3. Use mktemp instead of guessing a temp filename (avoids race conditions/symlink attacks) tmpfile=$(mktemp) || exit 1 trap 'rm -f "$tmpfile"' EXIT # 4. Be careful with $IFS and globbing when building commands from variables set -f # disable globbing temporarily if you don't need it (noglob) # 5. Don't hardcode secrets — read from environment or a secrets manager, not committed files api_key="${API_KEY:?Set API_KEY environment variable}" # 6. Validate/whitelist before passing user input to other commands case "$user_choice" in start|stop|restart) systemctl "$user_choice" myapp.service ;; *) echo "Invalid action" >&2; exit 1 ;; esac # 7. Drop privileges when you can — don't run as root longer than necessary sudo -u appuser /opt/app/run.sh ``` ### Performance Tips ```bash # Avoid spawning external processes inside hot loops — prefer builtins # SLOW: a subprocess fork on every iteration for f in *.txt; do base=$(basename "$f"); done # FAST: pure Bash parameter expansion, no fork for f in *.txt; do base="${f##*/}"; done # Batch operations instead of looping where possible # SLOW: while read -r line; do grep "$line" bigfile.txt; done < patterns.txt # FAST: let grep itself search for multiple patterns in one pass grep -F -f patterns.txt bigfile.txt # Use `mapfile`/`readarray` to slurp a file once instead of repeated reads mapfile -t lines < file.txt # Profile a script's runtime time ./script.sh TIMEFORMAT='Elapsed: %R seconds'; time my_function ``` `shellcheck` should run clean (or with explicitly justified `# shellcheck disable=SCxxxx` comments) before any script reaches production — it is the single highest-leverage tool for catching the security/correctness issues above automatically. --- # Part IX — Real-World Projects ## Chapter 21 — Four Complete Projects These four scripts combine everything from the previous chapters into deployable, production-style tools. Each follows the professional conventions established above: header docs, `set -euo pipefail`, trapped cleanup, logging, and meaningful exit codes. ### Project 1 — System Health Check & Monitor ```bash #!/usr/bin/env bash ############################################################################### # health_check.sh — reports system health and exits non-zero if any # threshold is breached. Designed to be run from cron/systemd and alert # (e.g., via the exit code, or by piping output to a notifier). ############################################################################### set -euo pipefail CPU_THRESHOLD=90 MEM_THRESHOLD=90 DISK_THRESHOLD=85 EXIT_CODE=0 check() { local label="$1" value="$2" threshold="$3" if (( value >= threshold )); then echo "[CRITICAL] $label at ${value}% (threshold ${threshold}%)" EXIT_CODE=2 else echo "[OK] $label at ${value}%" fi } cpu_idle=$(top -bn1 | awk -F'id,' '{split($1,a," "); print a[length(a)]}') cpu_usage=$(awk -v idle="$cpu_idle" 'BEGIN { printf "%d", 100 - idle }') check "CPU usage" "$cpu_usage" "$CPU_THRESHOLD" mem_total=$(awk '/MemTotal/ {print $2}' /proc/meminfo) mem_avail=$(awk '/MemAvailable/ {print $2}' /proc/meminfo) mem_usage=$(( 100 * (mem_total - mem_avail) / mem_total )) check "Memory usage" "$mem_usage" "$MEM_THRESHOLD" disk_usage=$(df / | awk 'NR==2 {gsub("%",""); print $5}') check "Disk usage (/)" "$disk_usage" "$DISK_THRESHOLD" exit "$EXIT_CODE" ``` ### Project 2 — Automated Backup with Rotation ```bash #!/usr/bin/env bash ############################################################################### # backup.sh — tars up a directory, timestamps it, uploads is optional, # and deletes backups older than RETENTION_DAYS. Intended for a daily cron job. # Usage: ./backup.sh /path/to/source /path/to/backup-dir [retention_days] ############################################################################### set -euo pipefail SRC="${1:?Usage: $0 [retention_days]}" DEST="${2:?Usage: $0 [retention_days]}" RETENTION_DAYS="${3:-7}" TIMESTAMP=$(date +%Y%m%d-%H%M%S) ARCHIVE="$DEST/backup-${TIMESTAMP}.tar.gz" LOCKFILE="/tmp/backup.lock" exec 200> "$LOCKFILE" flock -n 200 || { echo "Backup already running. Exiting." >&2; exit 1; } log() { printf '%s %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*"; } mkdir -p "$DEST" log "Starting backup of $SRC -> $ARCHIVE" tar -czf "$ARCHIVE" -C "$(dirname "$SRC")" "$(basename "$SRC")" log "Backup created: $(du -h "$ARCHIVE" | cut -f1)" log "Pruning backups older than $RETENTION_DAYS days" find "$DEST" -name "backup-*.tar.gz" -mtime "+${RETENTION_DAYS}" -print -delete log "Backup complete. Current backups:" ls -lh "$DEST"/backup-*.tar.gz ``` ### Project 3 — Log Analyzer ```bash #!/usr/bin/env bash ############################################################################### # log_analyzer.sh — summarizes an Nginx/Apache-style access log: # total requests, top IPs, top paths, status code breakdown, error rate. # Usage: ./log_analyzer.sh /var/log/nginx/access.log ############################################################################### set -euo pipefail LOGFILE="${1:?Usage: $0 }" [[ -r "$LOGFILE" ]] || { echo "Cannot read $LOGFILE" >&2; exit 1; } total=$(wc -l < "$LOGFILE") echo "=== Log Analysis: $LOGFILE ===" echo "Total requests: $total" echo -e "\n--- Top 10 IPs ---" awk '{print $1}' "$LOGFILE" | sort | uniq -c | sort -rn | head -10 echo -e "\n--- Top 10 Requested Paths ---" awk -F'"' '{print $2}' "$LOGFILE" | awk '{print $2}' | sort | uniq -c | sort -rn | head -10 echo -e "\n--- Status Code Breakdown ---" awk -F'"' '{print $3}' "$LOGFILE" | awk '{print $1}' | sort | uniq -c | sort -rn errors=$(awk -F'"' '{print $3}' "$LOGFILE" | awk '$1 ~ /^5/' | wc -l) error_rate=$(awk -v e="$errors" -v t="$total" 'BEGIN { printf "%.2f", (e/t)*100 }') echo -e "\n--- Error Rate ---" echo "5xx errors: $errors / $total ($error_rate%)" ``` ### Project 4 — Deployment Script ```bash #!/usr/bin/env bash ############################################################################### # deploy.sh — pulls latest code, runs a build, swaps a symlink for # zero-downtime cutover, and rolls back automatically on health-check failure. # Usage: ./deploy.sh ############################################################################### set -euo pipefail ENVIRONMENT="${1:?Usage: $0 }" APP_DIR="/opt/myapp" RELEASES_DIR="$APP_DIR/releases" CURRENT_LINK="$APP_DIR/current" RELEASE_ID=$(date +%Y%m%d%H%M%S) RELEASE_DIR="$RELEASES_DIR/$RELEASE_ID" HEALTH_URL="http://localhost:8080/health" PREVIOUS_RELEASE="" log() { printf '[%s] %s\n' "$(date '+%H:%M:%S')" "$*"; } rollback() { log "DEPLOY FAILED — rolling back to previous release" if [[ -n "$PREVIOUS_RELEASE" ]]; then ln -sfn "$PREVIOUS_RELEASE" "$CURRENT_LINK" systemctl restart myapp.service log "Rolled back to $PREVIOUS_RELEASE" fi exit 1 } trap rollback ERR [[ -L "$CURRENT_LINK" ]] && PREVIOUS_RELEASE=$(readlink -f "$CURRENT_LINK") log "Deploying release $RELEASE_ID to $ENVIRONMENT" mkdir -p "$RELEASE_DIR" git clone --depth 1 --branch "$ENVIRONMENT" git@example.com:org/myapp.git "$RELEASE_DIR" cd "$RELEASE_DIR" log "Installing dependencies & building" npm ci --omit=dev npm run build log "Switching symlink to new release" ln -sfn "$RELEASE_DIR" "$CURRENT_LINK" systemctl restart myapp.service log "Waiting for health check" for i in {1..10}; do if curl -sf "$HEALTH_URL" > /dev/null; then log "Health check passed. Deployment successful." trap - ERR # disable rollback trap -- we succeeded # Keep only the last 5 releases ls -dt "$RELEASES_DIR"/*/ | tail -n +6 | xargs -r rm -rf exit 0 fi sleep 3 done log "Health check never passed" exit 1 # triggers the ERR trap -> rollback ``` --- # Appendix A — Bash Cheat Sheet ### Shebang & Safety Header ```bash #!/usr/bin/env bash set -euo pipefail IFS=$'\n\t' ``` ### Quoting | | | |---|---| | `"$var"` | always quote expansions | | `'literal'` | no expansion at all | | `"$@"` | preserve args as separate words | ### Tests | Expr | Meaning | |---|---| | `[[ -f f ]]` | regular file exists | | `[[ -d d ]]` | directory exists | | `[[ -z s ]]` / `[[ -n s ]]` | empty / non-empty string | | `[[ a == b ]]` | string equality | | `(( a == b ))` | numeric equality | | `[[ s =~ regex ]]` | regex match | ### Parameter Expansion | Form | Meaning | |---|---| | `${v:-d}` | default if unset/empty | | `${v:=d}` | default + assign | | `${v:?msg}` | error if unset/empty | | `${#v}` | length | | `${v:off:len}` | substring | | `${v/x/y}` / `${v//x/y}` | replace first / all | | `${v#p}` / `${v##p}` | strip shortest/longest prefix | | `${v%p}` / `${v%%p}` | strip shortest/longest suffix | | `${v^^}` / `${v,,}` | uppercase / lowercase all | ### Loops ```bash for x in list; do :; done for ((i=0;i f` / `>> f` | stdout overwrite / append | | `2> f` | stderr | | `&> f` | stdout+stderr | | `< f` | stdin from file | | `<<< "s"` | here-string | | `<