Shell Patterns

1. Why Shell Patterns Matter

Shell scripting isn’t just about writing scripts. It’s about:

  • Constructing powerful one-liners for ad-hoc tasks

  • Understanding command output you’ll see in production

  • Writing automation that won’t break on edge cases

  • Troubleshooting scripts you didn’t write

This guide focuses on Bash. While most patterns work in zsh, ksh, and other Bourne-compatible shells, subtle differences exist. When in doubt, check with echo $SHELL and $BASH_VERSION.

2. Heredoc (Here Documents)

Heredoc allows multi-line input to commands without external files.

2.1. Basic Syntax

╔═══════════════════════════════════════════════════════════════════════════╗
║                          HEREDOC SYNTAX                                    ║
╠═══════════════════════════════════════════════════════════════════════════╣
║                                                                            ║
║  command << DELIMITER                                                      ║
║  line 1                                                                    ║
║  line 2                                                                    ║
║  line 3                                                                    ║
║  DELIMITER                                                                 ║
║                                                                            ║
║  RULES:                                                                    ║
║  • DELIMITER can be any word (EOF, END, DOC are common)                    ║
║  • Closing DELIMITER must be ALONE on its line (no leading spaces*)        ║
║  • Content between delimiters is passed as stdin to command                ║
║                                                                            ║
║  *Exception: <<-DELIMITER allows leading TABS (not spaces) in content      ║
║   and on the closing delimiter itself                                      ║
║                                                                            ║
╚═══════════════════════════════════════════════════════════════════════════╝

2.2. Expansion Control: Quoted vs Unquoted Delimiter

This is CRITICAL to understand:

# UNQUOTED delimiter - variables ARE expanded
cat << EOF
Hello $USER
Home: $HOME
Today: $(date)
EOF
# Output: Hello evanusmodestus
#         Home: /home/evanusmodestus
#         Today: Sun Feb  9 14:32:15 PST 2026

# QUOTED delimiter - variables NOT expanded (literal)
cat << 'EOF'
Hello $USER
Home: $HOME
Today: $(date)
EOF
# Output: Hello $USER
#         Home: $HOME
#         Today: $(date)

Rule of thumb: Use 'EOF' (quoted) when you want literal content - no variable expansion, no command substitution. This is what you want 90% of the time for config files and scripts.

2.3. Tab Stripping with <←

The <← variant strips leading TABS (not spaces) from content:

# Without <<- (tabs visible in output)
if true; then
    cat << EOF
    This has leading spaces
EOF
fi

# With <<- (tabs stripped)
if true; then
	cat <<- EOF
	This content is indented with tabs
	They're stripped from output
	EOF
fi

This allows heredocs inside indented code blocks while producing clean output.

2.4. Heredoc Use Cases

2.4.1. Git Commit Messages

git commit -m "$(cat << 'EOF'
docs(domain-join): Update DC hostname

- Changed dc-01 to home-dc01
- Added DC migration section
- Updated krb5.conf examples
EOF
)"

2.4.2. Configuration File Generation

# Create entire config file
sudo tee /etc/krb5.conf > /dev/null << 'EOF'
[libdefaults]
    default_realm = INSIDE.DOMUSDIGITALIS.DEV
    dns_lookup_realm = false
    dns_lookup_kdc = true

[realms]
    INSIDE.DOMUSDIGITALIS.DEV = {
        kdc = home-dc01.inside.domusdigitalis.dev
        admin_server = home-dc01.inside.domusdigitalis.dev
    }

[domain_realm]
    .inside.domusdigitalis.dev = INSIDE.DOMUSDIGITALIS.DEV
    inside.domusdigitalis.dev = INSIDE.DOMUSDIGITALIS.DEV
EOF

2.4.3. gopass Multi-Line Insert

gopass insert ADMINISTRATIO/servers/home-dc01/meta << 'EOF'
hostname: home-dc01
ip: 10.50.1.50
os: Windows Server 2025 Core
domain: inside.domusdigitalis.dev
roles: AD DS, DNS
deployed: 2026-02-09
notes: New forest, replaced old dc-01
EOF

2.4.4. SSH Remote Commands

# Execute multiple commands on remote host
ssh user@server << 'EOF'
cd /var/log
grep -i error syslog | tail -20
df -h
uptime
EOF

2.4.5. SQL Queries

mysql -u admin -p"$PASSWORD" << 'EOF'
SELECT id, username, last_login
FROM users
WHERE last_login < DATE_SUB(NOW(), INTERVAL 90 DAY)
ORDER BY last_login;
EOF

2.4.6. Script Generation

# Generate a script dynamically
cat > /tmp/backup.sh << 'EOF'
#!/bin/bash
set -euo pipefail

BACKUP_DIR="/backup/$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

for dir in /etc /home /var/log; do
    tar -czf "$BACKUP_DIR/$(basename "$dir").tar.gz" "$dir"
done

echo "Backup complete: $BACKUP_DIR"
EOF
chmod +x /tmp/backup.sh

2.4.7. With Variable Interpolation (Unquoted)

HOSTNAME="modestus-aw"
DOMAIN="inside.domusdigitalis.dev"

# Variables are expanded because EOF is not quoted
sudo tee /etc/hostname > /dev/null << EOF
${HOSTNAME}.${DOMAIN}
EOF

2.5. Herestring (<<<)

For single-line input, use herestring:

# Instead of: echo "hello" | command
# Use:        command <<< "hello"

# Example: Base64 encode a string
base64 <<< "secret password"

# Example: Read fields
read -r user host <<< "admin server01"
echo "User: $user, Host: $host"

# Variables are expanded
base64 <<< "User is $USER"

3. Brace Expansion

Brace expansion generates multiple strings from a pattern. It happens BEFORE variable expansion.

3.1. Basic Patterns

# Comma-separated list
echo {a,b,c}
# Output: a b c

echo file{1,2,3}.txt
# Output: file1.txt file2.txt file3.txt

# Nested braces
echo {a,b}{1,2}
# Output: a1 a2 b1 b2

# Empty element
echo file{,-backup}.txt
# Output: file.txt file-backup.txt

3.2. Sequences

# Numeric range
echo {1..5}
# Output: 1 2 3 4 5

# Alphabetic range
echo {a..e}
# Output: a b c d e

# With padding
echo {01..10}
# Output: 01 02 03 04 05 06 07 08 09 10

# With step
echo {0..10..2}
# Output: 0 2 4 6 8 10

# Reverse
echo {5..1}
# Output: 5 4 3 2 1

# Combined with text
echo file{01..05}.log
# Output: file01.log file02.log file03.log file04.log file05.log

3.3. Production Use Cases

3.3.1. Create Multiple Directories

mkdir -p project/{src,docs,tests,config}/{main,backup}
# Creates:
# project/src/main, project/src/backup
# project/docs/main, project/docs/backup
# project/tests/main, project/tests/backup
# project/config/main, project/config/backup

3.3.2. Backup Before Edit

# Copy file to .bak extension
cp /etc/krb5.conf{,.bak}
# Equivalent to: cp /etc/krb5.conf /etc/krb5.conf.bak

# With timestamp
cp /etc/krb5.conf{,.$(date +%Y%m%d)}
# Creates: /etc/krb5.conf.20260209

3.3.3. Rename/Move Files

# Change extension
mv file.{txt,md}
# Equivalent to: mv file.txt file.md

# Change directory
mv /old/path/file.conf{,} /new/path/
# Wait, that doesn't work. Use:
mv /old/path/file.conf /new/path/

3.3.4. Copy Multiple SSH Keys

# Copy several related files
cat ~/.ssh/id_ed25519_{d000,sk_rk_d000,sk_rk_d000_secondary}.pub | wl-copy

3.3.5. Create Test Files

# Create 100 test files
touch test_file_{001..100}.txt

# Create files for each day of January
touch log_2026-01-{01..31}.txt

3.3.6. Download Sequential URLs

# Download pages 1-10
curl -O "https://example.com/page[1-10].html"

# Note: curl has its own range syntax, but brace expansion works too:
for i in {1..10}; do
    curl -O "https://example.com/page${i}.html"
done

3.4. Brace Expansion vs Globs

Brace expansion generates strings. It doesn’t check if files exist.

Glob patterns match existing files. *.txt only matches files that exist.

# Brace expansion: generates strings (files may not exist)
echo {a,b,c}.txt
# Output: a.txt b.txt c.txt (always)

# Glob: matches existing files
echo *.txt
# Output: (only files that actually exist with .txt extension)

4. Parameter Expansion

Parameter expansion is the shell’s Swiss Army knife for string manipulation.

4.1. Basic Forms

# Simple expansion
echo $VAR
echo ${VAR}    # Preferred - clearer boundaries

# Required: when followed by valid identifier characters
echo ${VAR}able    # Correct: "valuable" (if VAR=valu)
echo $VAR_able     # Wrong: expands $VAR_able, not $VAR

4.2. Default Values

╔═══════════════════════════════════════════════════════════════════════════╗
║                        DEFAULT VALUE OPERATORS                             ║
╠═══════════════════════════════════════════════════════════════════════════╣
║                                                                            ║
║  ${VAR:-default}   Use default if VAR is unset OR empty                   ║
║  ${VAR-default}    Use default if VAR is unset (empty is valid)           ║
║                                                                            ║
║  ${VAR:=default}   Set VAR to default if unset OR empty, then use        ║
║  ${VAR=default}    Set VAR to default if unset (empty is valid)          ║
║                                                                            ║
║  ${VAR:+value}     Use value if VAR is set AND non-empty                  ║
║  ${VAR+value}      Use value if VAR is set (even if empty)                ║
║                                                                            ║
║  ${VAR:?error}     Exit with error if VAR is unset OR empty               ║
║  ${VAR?error}      Exit with error if VAR is unset                        ║
║                                                                            ║
║  KEY INSIGHT: The colon (:) means "also treat empty as unset"             ║
║                                                                            ║
╚═══════════════════════════════════════════════════════════════════════════╝
# Use default if unset
echo "${NAME:-anonymous}"

# Set and use default
LOGDIR="${LOGDIR:=/var/log/myapp}"
echo "$LOGDIR"

# Require variable (exit if missing)
: "${CONFIG_FILE:?ERROR: CONFIG_FILE must be set}"

# Conditional value
echo "Debug mode: ${DEBUG:+enabled}"

4.3. String Length

VAR="hello world"
echo ${#VAR}
# Output: 11

4.4. Substring Extraction

VAR="hello world"

# From position (0-indexed)
echo ${VAR:6}
# Output: world

# From position with length
echo ${VAR:0:5}
# Output: hello

# Negative offset (from end) - note the space or parentheses
echo ${VAR: -5}
# Output: world

echo ${VAR:(-5)}
# Output: world

# Last 5 characters
echo ${VAR: -5:3}
# Output: wor

4.5. Pattern Removal

╔═══════════════════════════════════════════════════════════════════════════╗
║                        PATTERN REMOVAL OPERATORS                           ║
╠═══════════════════════════════════════════════════════════════════════════╣
║                                                                            ║
║  ${VAR#pattern}    Remove shortest match from BEGINNING                   ║
║  ${VAR##pattern}   Remove longest match from BEGINNING                    ║
║                                                                            ║
║  ${VAR%pattern}    Remove shortest match from END                         ║
║  ${VAR%%pattern}   Remove longest match from END                          ║
║                                                                            ║
║  MEMORY AID:                                                               ║
║  # is on the LEFT side of $ on keyboard → removes from LEFT (beginning)   ║
║  % is on the RIGHT side of $ on keyboard → removes from RIGHT (end)       ║
║                                                                            ║
╚═══════════════════════════════════════════════════════════════════════════╝
FILE="/path/to/document.txt"

# Get filename (remove directory path)
echo ${FILE##*/}
# Output: document.txt

# Get directory (remove filename)
echo ${FILE%/*}
# Output: /path/to

# Get extension
echo ${FILE##*.}
# Output: txt

# Remove extension
echo ${FILE%.*}
# Output: /path/to/document

# Remove all extensions (e.g., file.tar.gz)
echo ${FILE%%.*}
# Output: /path/to/document (stops at first dot from right)
# Wait, that's wrong. Let me fix:
FILE2="/path/to/archive.tar.gz"
echo ${FILE2%%.*}
# Output: /path/to/archive

4.6. Pattern Replacement

VAR="hello world world"

# Replace first occurrence
echo ${VAR/world/universe}
# Output: hello universe world

# Replace all occurrences
echo ${VAR//world/universe}
# Output: hello universe universe

# Replace at beginning (like sed ^)
echo ${VAR/#hello/goodbye}
# Output: goodbye world world

# Replace at end (like sed $)
echo ${VAR/%world/universe}
# Output: hello world universe

# Delete (replace with nothing)
echo ${VAR//world}
# Output: hello

4.7. Case Modification (Bash 4.0+)

VAR="Hello World"

# Lowercase first character
echo ${VAR,}
# Output: hello World

# Lowercase all
echo ${VAR,,}
# Output: hello world

# Uppercase first character
echo ${VAR^}
# Output: Hello World

# Uppercase all
echo ${VAR^^}
# Output: HELLO WORLD

# Toggle case
echo ${VAR~~}
# Output: hELLO wORLD

4.8. Indirect Expansion

NAME="greeting"
greeting="Hello, World!"

# Indirect: get value of variable whose name is in NAME
echo ${!NAME}
# Output: Hello, World!

# Useful for dynamic variable names
SERVER="prod"
prod_host="10.50.1.100"
dev_host="10.50.1.200"

echo "Connecting to ${!SERVER@}_host"
# Hmm, that's not quite right. Let me fix:

VAR_NAME="${SERVER}_host"
echo "Connecting to ${!VAR_NAME}"
# Output: Connecting to 10.50.1.100

4.9. Array Operations via Parameter Expansion

arr=(one two three four five)

# All elements
echo "${arr[@]}"
# Output: one two three four five

# Number of elements
echo "${#arr[@]}"
# Output: 5

# Keys/indices
echo "${!arr[@]}"
# Output: 0 1 2 3 4

# Slice: from index 2, take 2 elements
echo "${arr[@]:2:2}"
# Output: three four

5. Arrays

Bash supports indexed (0-based) and associative (key-value) arrays.

5.1. Indexed Arrays

# Declaration methods
arr=(one two three)           # Inline
arr[0]="one"                  # Individual assignment
declare -a arr                # Explicit declaration
readarray -t arr < file.txt   # From file (one element per line)

# Access elements
echo "${arr[0]}"              # First element
echo "${arr[-1]}"             # Last element (Bash 4.3+)
echo "${arr[@]}"              # All elements (preserves word boundaries)
echo "${arr[*]}"              # All elements (as single word)

# Array length
echo "${#arr[@]}"             # Number of elements
echo "${#arr[0]}"             # Length of first element

# Indices
echo "${!arr[@]}"             # All indices

# Append
arr+=("four")                 # Append element
arr+=(five six)               # Append multiple

# Delete
unset 'arr[1]'                # Delete element (leaves gap!)
arr=("${arr[@]}")             # Reindex to close gaps

# Slice
echo "${arr[@]:1:2}"          # Elements 1 and 2 (0-indexed)

# Copy
new_arr=("${arr[@]}")

5.2. Associative Arrays (Bash 4.0+)

# Must declare explicitly
declare -A servers

# Assign
servers[web]="10.50.1.10"
servers[db]="10.50.1.20"
servers[cache]="10.50.1.30"

# Or inline
declare -A servers=(
    [web]="10.50.1.10"
    [db]="10.50.1.20"
    [cache]="10.50.1.30"
)

# Access
echo "${servers[web]}"        # Get value
echo "${servers[@]}"          # All values
echo "${!servers[@]}"         # All keys

# Check if key exists
if [[ -v servers[web] ]]; then
    echo "web server defined"
fi

# Iterate
for server in "${!servers[@]}"; do
    echo "$server: ${servers[$server]}"
done

# Delete key
unset 'servers[cache]'

5.3. Array Iteration

arr=(one two "three four" five)

# CORRECT: Quoted expansion preserves elements
for item in "${arr[@]}"; do
    echo "Item: $item"
done
# Output:
# Item: one
# Item: two
# Item: three four
# Item: five

# WRONG: Unquoted splits on whitespace
for item in ${arr[@]}; do
    echo "Item: $item"
done
# Output:
# Item: one
# Item: two
# Item: three
# Item: four
# Item: five

5.4. Array Processing Patterns

# Filter array
arr=(1 2 3 4 5 6 7 8 9 10)
evens=()
for n in "${arr[@]}"; do
    (( n % 2 == 0 )) && evens+=("$n")
done
echo "${evens[@]}"
# Output: 2 4 6 8 10

# Map/transform array
files=(file1.txt file2.txt file3.txt)
basenames=()
for f in "${files[@]}"; do
    basenames+=("${f%.txt}")
done
echo "${basenames[@]}"
# Output: file1 file2 file3

# Join array to string
IFS=','
echo "${arr[*]}"
# Output: 1,2,3,4,5,6,7,8,9,10
unset IFS

# Split string to array
str="one,two,three,four"
IFS=',' read -ra arr <<< "$str"
echo "${arr[1]}"
# Output: two

6. Command Substitution

Capture command output as a string.

6.1. Syntax

# Modern (preferred)
result=$(command)

# Legacy (avoid - can't nest easily)
result=`command`

# Nested substitution
result=$(echo "Date: $(date +%Y-%m-%d)")

6.2. Use Cases

# Capture command output
files=$(ls -1 *.txt)
count=$(wc -l < file.txt)
hostname=$(hostname -f)

# Use in strings
echo "Current user: $(whoami) on $(hostname)"

# Use in conditionals
if [[ $(whoami) == "root" ]]; then
    echo "Running as root"
fi

# Use in arithmetic
file_count=$(ls -1 | wc -l)
(( file_count > 10 )) && echo "Many files"

# Capture exit status AND output
if output=$(command 2>&1); then
    echo "Success: $output"
else
    echo "Failed: $output"
fi

6.3. Pitfalls

# WRONG: Unquoted loses whitespace
files=$(ls -1)
echo $files      # All on one line!

# CORRECT: Quote to preserve
echo "$files"    # Preserves newlines

# WRONG: Command substitution in single quotes
echo 'Today is $(date)'
# Output: Today is $(date)

# CORRECT: Use double quotes
echo "Today is $(date)"
# Output: Today is Sun Feb  9 14:32:15 PST 2026

7. Process Substitution

Process substitution provides command output as a file-like object.

7.1. Syntax

# Input: <(command)
# Treats command output as a file you can read

# Output: >(command)
# Treats command input as a file you can write to

7.2. Use Cases

# Compare output of two commands
diff <(ls /etc) <(ls /usr/etc)

# Compare sorted outputs
diff <(sort file1.txt) <(sort file2.txt)

# Feed command output to programs requiring filename
while read -r line; do
    echo "Processing: $line"
done < <(find /var/log -name "*.log" -mtime -1)

# Multiple inputs
paste <(cut -f1 file1.txt) <(cut -f1 file2.txt)

# Tee to multiple destinations
command | tee >(grep ERROR > errors.log) >(grep WARN > warnings.log) > full.log

# Write to multiple processes
echo "test" | tee >(md5sum) >(sha256sum) > /dev/null

7.3. Process Substitution vs Pipes

# With pipe, while loop runs in subshell - variables don't persist
count=0
cat file.txt | while read -r line; do
    ((count++))
done
echo "$count"  # Still 0!

# With process substitution, while loop runs in current shell
count=0
while read -r line; do
    ((count++))
done < <(cat file.txt)
echo "$count"  # Correct count!

# Or use lastpipe option (Bash 4.2+)
shopt -s lastpipe
count=0
cat file.txt | while read -r line; do
    ((count++))
done
echo "$count"  # Now works!

8. Clipboard Operations

8.1. Wayland (Modern Linux)

# Copy to clipboard
echo "text" | wl-copy

# Copy file contents
wl-copy < file.txt
cat file.txt | wl-copy

# Copy command output
ls -la | wl-copy
date | wl-copy

# Paste from clipboard
wl-paste

# Paste to file
wl-paste > output.txt

# Copy without newline (for passwords)
printf '%s' "password" | wl-copy

# Copy primary selection (middle-click)
echo "text" | wl-copy --primary
wl-paste --primary

# Watch clipboard and execute on change
wl-paste --watch cat    # Prints each new clipboard entry

8.2. X11 (xclip/xsel)

# xclip
echo "text" | xclip -selection clipboard
xclip -selection clipboard -o

# xsel
echo "text" | xsel --clipboard --input
xsel --clipboard --output

# Note: X11 has three selections:
# - PRIMARY (middle-click paste)
# - SECONDARY (rarely used)
# - CLIPBOARD (Ctrl+C/Ctrl+V)

8.3. gopass Integration

# Built-in clipboard copy (auto-clears after 45s)
gopass show -c ADMINISTRATIO/servers/home-dc01/Administrator

# Manual clipboard copy via wl-copy
gopass show -o ADMINISTRATIO/servers/home-dc01/Administrator | wl-copy

# Copy and keep (don't auto-clear)
gopass show -o PATH/TO/SECRET | wl-copy -o

9. Conditionals and Tests

9.1. Test Commands

# Three equivalent syntaxes:
test -f /etc/passwd
[ -f /etc/passwd ]
[[ -f /etc/passwd ]]    # Preferred (Bash extension)

9.2. [[ ]] vs [ ]

╔═══════════════════════════════════════════════════════════════════════════╗
║                        [[ ]] vs [ ] COMPARISON                             ║
╠═══════════════════════════════════════════════════════════════════════════╣
║                                                                            ║
║  [ ] (test)           [[ ]] (Bash keyword)                                ║
║  ─────────────────    ──────────────────────                               ║
║  POSIX compatible     Bash/Zsh/Ksh only                                   ║
║  Word splitting YES   Word splitting NO                                    ║
║  Glob expansion YES   Glob expansion NO                                    ║
║  Must quote vars      Quoting optional*                                    ║
║  -a for AND           && for AND                                           ║
║  -o for OR            || for OR                                            ║
║  No pattern match     Supports =~ regex                                    ║
║  No glob patterns     Supports == with globs                               ║
║                                                                            ║
║  *Still good practice to quote in [[                                       ║
║                                                                            ║
╚═══════════════════════════════════════════════════════════════════════════╝
# With [ ], unquoted variable can break
file="my file.txt"
[ -f $file ]      # Error: too many arguments
[ -f "$file" ]    # Works

# With [[ ]], safe without quotes (but quote anyway)
[[ -f $file ]]    # Works
[[ -f "$file" ]]  # Also works, preferred

9.3. File Tests

[[ -e "$file" ]]    # Exists (any type)
[[ -f "$file" ]]    # Regular file
[[ -d "$file" ]]    # Directory
[[ -L "$file" ]]    # Symbolic link
[[ -r "$file" ]]    # Readable
[[ -w "$file" ]]    # Writable
[[ -x "$file" ]]    # Executable
[[ -s "$file" ]]    # Non-empty (size > 0)
[[ -b "$file" ]]    # Block device
[[ -c "$file" ]]    # Character device
[[ -p "$file" ]]    # Named pipe (FIFO)
[[ -S "$file" ]]    # Socket
[[ -O "$file" ]]    # Owned by current user
[[ -G "$file" ]]    # Owned by current group

# File comparisons
[[ "$f1" -nt "$f2" ]]    # f1 newer than f2
[[ "$f1" -ot "$f2" ]]    # f1 older than f2
[[ "$f1" -ef "$f2" ]]    # Same inode (hard link)

9.4. String Tests

[[ -z "$str" ]]         # Empty (zero length)
[[ -n "$str" ]]         # Non-empty
[[ "$str" ]]            # Non-empty (shorthand)

[[ "$a" == "$b" ]]      # Equal
[[ "$a" != "$b" ]]      # Not equal
[[ "$a" < "$b" ]]       # Less than (lexicographic)
[[ "$a" > "$b" ]]       # Greater than (lexicographic)

# Pattern matching (glob)
[[ "$str" == *.txt ]]   # Ends with .txt
[[ "$str" == [0-9]* ]]  # Starts with digit

# Regex matching
[[ "$str" =~ ^[0-9]+$ ]]    # All digits
[[ "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]  # Email-ish

9.5. Numeric Tests

# Integer comparison (use (( )) or these operators)
[[ "$a" -eq "$b" ]]     # Equal
[[ "$a" -ne "$b" ]]     # Not equal
[[ "$a" -lt "$b" ]]     # Less than
[[ "$a" -le "$b" ]]     # Less than or equal
[[ "$a" -gt "$b" ]]     # Greater than
[[ "$a" -ge "$b" ]]     # Greater than or equal

# Arithmetic context (preferred for numbers)
(( a == b ))
(( a != b ))
(( a < b ))
(( a <= b ))
(( a > b ))
(( a >= b ))
(( a && b ))            # Both non-zero
(( a || b ))            # Either non-zero

9.6. Combining Tests

# AND
[[ -f "$file" && -r "$file" ]]

# OR
[[ -z "$var" || "$var" == "default" ]]

# NOT
[[ ! -f "$file" ]]

# Grouping
[[ (-f "$file" && -r "$file") || "$force" == "true" ]]

9.7. Variable Tests

# Variable is set (even if empty)
[[ -v var ]]

# Variable is set and is a nameref
[[ -R var ]]

# Variable is set to non-empty value
[[ -n "${var:-}" ]]
[[ "${var:-}" ]]

10. Arithmetic

10.1. Arithmetic Context

# Assignment
(( x = 5 ))
(( y = x + 10 ))

# Increment/decrement
(( x++ ))
(( x-- ))
(( x += 5 ))

# All C operators work
(( x = (5 + 3) * 2 ))    # 16
(( x = 10 / 3 ))         # 3 (integer division)
(( x = 10 % 3 ))         # 1 (modulo)
(( x = 2 ** 8 ))         # 256 (exponentiation)

# Comparison (returns 0 for true, 1 for false)
(( 5 > 3 )) && echo "yes"

# Ternary
(( x = (a > b) ? a : b ))

# Bitwise
(( x = 5 & 3 ))          # AND: 1
(( x = 5 | 3 ))          # OR: 7
(( x = 5 ^ 3 ))          # XOR: 6
(( x = ~5 ))             # NOT: -6
(( x = 1 << 4 ))         # Left shift: 16
(( x = 16 >> 2 ))        # Right shift: 4

10.2. $ Arithmetic Expansion

# Returns the value (use when you need the result as a string)
echo "Result: $(( 5 + 3 ))"
result=$(( x * 2 ))

# Variables don't need $ inside
x=5
echo $(( x + 10 ))       # 15
echo $(( $x + 10 ))      # Also works

# Common patterns
echo "Day $(( $(date +%d) + 1 ))"    # Tomorrow's day

10.3. let Command

# Alternative to (( ))
let "x = 5 + 3"
let "x++"
let "x += 10"

# Generally (( )) is preferred

11. Pattern Matching (Globs)

11.1. Basic Globs

*           # Any string (including empty)
?           # Any single character
[abc]       # Any one of a, b, c
[a-z]       # Any character in range
[!abc]      # Any character NOT a, b, c
[^abc]      # Same as [!abc]

# Examples
ls *.txt                    # All .txt files
ls file?.log                # file1.log, fileA.log, etc.
ls [Ff]ile.txt              # File.txt or file.txt
ls log.[0-9]                # log.0 through log.9

11.2. Extended Globs (extglob)

# Enable extended globs
shopt -s extglob

?(pattern)      # Zero or one occurrence
*(pattern)      # Zero or more occurrences
+(pattern)      # One or more occurrences
@(pattern)      # Exactly one occurrence
!(pattern)      # Anything EXCEPT pattern

# Examples
ls *.@(jpg|png|gif)         # Image files
ls !(*.txt)                 # All files except .txt
ls *.+(tar|gz)              # Files ending in tar or gz
rm *.!(keep)                # Remove all except .keep files

11.3. Glob Options

# Null glob: no match returns empty (not literal pattern)
shopt -s nullglob
files=(*.nonexistent)
echo "${#files[@]}"         # 0 (not 1)

# Fail glob: no match is an error
shopt -s failglob
ls *.nonexistent            # Error

# Dot glob: * matches dotfiles
shopt -s dotglob
ls *                        # Includes .bashrc, etc.

# Glob star: ** matches directories recursively
shopt -s globstar
ls **/*.txt                 # All .txt files in all subdirs

# No case glob: case-insensitive matching
shopt -s nocaseglob
ls *.TXT                    # Matches file.txt, FILE.TXT, etc.

12. Job Control

12.1. Background and Foreground

# Run command in background
long_command &

# List jobs
jobs
jobs -l     # With PIDs

# Bring to foreground
fg          # Most recent job
fg %1       # Job number 1
fg %command # Job starting with "command"

# Suspend foreground job
Ctrl+Z

# Resume in background
bg
bg %1

# Wait for background jobs
wait        # All jobs
wait %1     # Specific job
wait $!     # Last background process

12.2. Job Specification

%n          # Job number n
%string     # Job command starting with string
%?string    # Job command containing string
%%          # Current job
%+          # Current job (same as %%)
%-          # Previous job

12.3. Disown

# Remove job from shell's job table (survives logout)
long_command &
disown

# Or
disown %1

# Run immune to hangup from start
nohup long_command &

# Or with output redirection
nohup long_command > output.log 2>&1 &

13. Defensive Scripting Patterns

13.1. Strict Mode

#!/bin/bash
set -euo pipefail

# -e: Exit on error
# -u: Error on unset variables
# -o pipefail: Pipeline fails if any command fails

# Optional additions
set -E          # ERR trap inherited by functions
shopt -s inherit_errexit  # Command substitutions inherit -e

13.2. Variable Checking

# Require variable to be set
: "${CONFIG_FILE:?ERROR: CONFIG_FILE must be set}"

# Provide default
LOGDIR="${LOGDIR:-/var/log/myapp}"

# Check before use
if [[ -z "${API_KEY:-}" ]]; then
    echo "ERROR: API_KEY not set" >&2
    exit 1
fi

13.3. Command Existence

# Check if command exists
if ! command -v docker &> /dev/null; then
    echo "ERROR: docker not installed" >&2
    exit 1
fi

# Or
type jq &> /dev/null || { echo "jq required"; exit 1; }

# Don't use: which, whereis, hash (less reliable)

13.4. File Safety

# Check file before reading
if [[ ! -r "$CONFIG_FILE" ]]; then
    echo "Cannot read $CONFIG_FILE" >&2
    exit 1
fi

# Check directory before writing
if [[ ! -d "$OUTPUT_DIR" ]]; then
    mkdir -p "$OUTPUT_DIR" || exit 1
fi

# Temp file best practice
tmpfile=$(mktemp)
trap 'rm -f "$tmpfile"' EXIT

# Or with directory
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT

13.5. Error Handling

# Function for clean error messages
die() {
    echo "ERROR: $*" >&2
    exit 1
}

# Usage
[[ -f "$file" ]] || die "File not found: $file"

# With cleanup
cleanup() {
    rm -f "$tmpfile"
    # Other cleanup
}
trap cleanup EXIT

# Catch and handle errors
if ! output=$(command 2>&1); then
    echo "Command failed: $output" >&2
    exit 1
fi

14. Production Scenarios

14.1. Scenario 1: Backup Script with Safety

#!/bin/bash
set -euo pipefail

# Configuration with defaults
BACKUP_DIR="${BACKUP_DIR:-/backup}"
RETENTION_DAYS="${RETENTION_DAYS:-30}"
declare -a DIRS_TO_BACKUP=(/etc /home /var/log)

# Require root
[[ $EUID -eq 0 ]] || { echo "Must run as root" >&2; exit 1; }

# Verify backup destination
[[ -d "$BACKUP_DIR" ]] || mkdir -p "$BACKUP_DIR"

# Create dated backup directory
DATE=$(date +%Y%m%d_%H%M%S)
CURRENT_BACKUP="$BACKUP_DIR/$DATE"
mkdir -p "$CURRENT_BACKUP"

# Backup each directory
for dir in "${DIRS_TO_BACKUP[@]}"; do
    if [[ -d "$dir" ]]; then
        name="${dir//\//_}"
        tar -czf "$CURRENT_BACKUP/${name}.tar.gz" "$dir"
        echo "Backed up: $dir"
    else
        echo "WARNING: $dir does not exist, skipping" >&2
    fi
done

# Cleanup old backups
find "$BACKUP_DIR" -maxdepth 1 -type d -mtime "+$RETENTION_DAYS" -exec rm -rf {} \;

echo "Backup complete: $CURRENT_BACKUP"

14.2. Scenario 2: Log Processing Pipeline

#!/bin/bash
set -euo pipefail

LOGFILE="${1:?Usage: $0 <logfile>}"

[[ -r "$LOGFILE" ]] || { echo "Cannot read $LOGFILE" >&2; exit 1; }

# Count errors by type using associative array
declare -A error_counts

while read -r line; do
    if [[ "$line" =~ ERROR:\ ([A-Z_]+) ]]; then
        error_type="${BASH_REMATCH[1]}"
        (( error_counts[$error_type]++ )) || true
    fi
done < "$LOGFILE"

# Output sorted by count
for error_type in "${!error_counts[@]}"; do
    echo "${error_counts[$error_type]} $error_type"
done | sort -rn

14.3. Scenario 3: Server Health Check

#!/bin/bash
set -euo pipefail

declare -A servers=(
    [web]="10.50.1.10"
    [db]="10.50.1.20"
    [cache]="10.50.1.30"
)

check_server() {
    local name="$1"
    local ip="$2"

    if ping -c 1 -W 2 "$ip" &> /dev/null; then
        echo "[OK] $name ($ip) is reachable"
        return 0
    else
        echo "[FAIL] $name ($ip) is unreachable"
        return 1
    fi
}

failed=0
for name in "${!servers[@]}"; do
    if ! check_server "$name" "${servers[$name]}"; then
        ((failed++))
    fi
done

(( failed == 0 )) || { echo "$failed server(s) unreachable"; exit 1; }
echo "All servers healthy"

14.4. Scenario 4: Configuration Template Expansion

#!/bin/bash
set -euo pipefail

# Variables for template
HOSTNAME="$(hostname -f)"
IP_ADDRESS="$(hostname -I | awk '{print $1}')"
DOMAIN="inside.domusdigitalis.dev"
DATE="$(date +%Y-%m-%d)"

# Template expansion using heredoc with unquoted delimiter
generate_config() {
    cat << EOF
# Generated on $DATE for $HOSTNAME
server {
    listen 443 ssl;
    server_name ${HOSTNAME};

    ssl_certificate /etc/ssl/certs/${HOSTNAME}.pem;
    ssl_certificate_key /etc/ssl/private/${HOSTNAME}.key;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
    }
}
EOF
}

# Note: \$host and \$remote_addr are escaped to output literal $
# while ${HOSTNAME} is expanded

generate_config > /etc/nginx/conf.d/"${HOSTNAME}".conf

15. Quick Reference

Pattern Description

Heredoc

<< 'EOF'

Heredoc, no expansion (literal content)

<< EOF

Heredoc, with variable expansion

<← EOF

Heredoc, strip leading tabs

<<< "string"

Herestring (single-line input)

Parameter Expansion

${VAR:-default}

Use default if unset/empty

${VAR:=default}

Set to default if unset/empty

${VAR:?error}

Error if unset/empty

${#VAR}

Length of string

${VAR#pattern}

Remove shortest match from start

${VAR##pattern}

Remove longest match from start

${VAR%pattern}

Remove shortest match from end

${VAR%%pattern}

Remove longest match from end

${VAR/old/new}

Replace first occurrence

${VAR//old/new}

Replace all occurrences

${VAR,,}

Lowercase all

${VAR^^}

Uppercase all

Brace Expansion

{a,b,c}

Generate: a b c

{1..10}

Generate: 1 2 3 …​ 10

{01..10}

Generate: 01 02 03 …​ 10 (padded)

{1..10..2}

Generate: 1 3 5 7 9 (step 2)

cp file{,.bak}

Copy file to file.bak

Arrays

arr=(a b c)

Create indexed array

${arr[0]}

First element

${arr[@]}

All elements (separate words)

${#arr[@]}

Array length

${!arr[@]}

All indices/keys

arr+=(d)

Append element

Substitution

$(command)

Command substitution

<(command)

Process substitution (input)

>(command)

Process substitution (output)

Tests

[[ -f file ]]

File exists and is regular file

[[ -d dir ]]

Directory exists

[[ -z "$str" ]]

String is empty

[[ "$a" == "$b" ]]

Strings equal

[[ "$s" =~ regex ]]

Regex match

a > b

Numeric comparison

Clipboard (Wayland)

cmd | wl-copy

Copy to clipboard

wl-paste

Paste from clipboard

16.2. Cross-Site References

  • gopass Password Manager - Credential management for infrastructure

  • Domain Join Guide - Heredoc and sed usage examples