Shell Patterns
1. Why Shell Patterns Matter
Shell scripting isn’t just about writing scripts. It’s about:
-
Constructing powerful one-liners for ad-hoc tasks
-
Understanding command output you’ll see in production
-
Writing automation that won’t break on edge cases
-
Troubleshooting scripts you didn’t write
|
This guide focuses on Bash. While most patterns work in zsh, ksh, and other Bourne-compatible shells, subtle differences exist. When in doubt, check with |
2. Heredoc (Here Documents)
Heredoc allows multi-line input to commands without external files.
2.1. Basic Syntax
╔═══════════════════════════════════════════════════════════════════════════╗
║ HEREDOC SYNTAX ║
╠═══════════════════════════════════════════════════════════════════════════╣
║ ║
║ command << DELIMITER ║
║ line 1 ║
║ line 2 ║
║ line 3 ║
║ DELIMITER ║
║ ║
║ RULES: ║
║ • DELIMITER can be any word (EOF, END, DOC are common) ║
║ • Closing DELIMITER must be ALONE on its line (no leading spaces*) ║
║ • Content between delimiters is passed as stdin to command ║
║ ║
║ *Exception: <<-DELIMITER allows leading TABS (not spaces) in content ║
║ and on the closing delimiter itself ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════╝
2.2. Expansion Control: Quoted vs Unquoted Delimiter
This is CRITICAL to understand:
# UNQUOTED delimiter - variables ARE expanded
cat << EOF
Hello $USER
Home: $HOME
Today: $(date)
EOF
# Output: Hello evanusmodestus
# Home: /home/evanusmodestus
# Today: Sun Feb 9 14:32:15 PST 2026
# QUOTED delimiter - variables NOT expanded (literal)
cat << 'EOF'
Hello $USER
Home: $HOME
Today: $(date)
EOF
# Output: Hello $USER
# Home: $HOME
# Today: $(date)
|
Rule of thumb: Use |
2.3. Tab Stripping with <←
The <← variant strips leading TABS (not spaces) from content:
# Without <<- (tabs visible in output)
if true; then
cat << EOF
This has leading spaces
EOF
fi
# With <<- (tabs stripped)
if true; then
cat <<- EOF
This content is indented with tabs
They're stripped from output
EOF
fi
This allows heredocs inside indented code blocks while producing clean output.
2.4. Heredoc Use Cases
2.4.1. Git Commit Messages
git commit -m "$(cat << 'EOF'
docs(domain-join): Update DC hostname
- Changed dc-01 to home-dc01
- Added DC migration section
- Updated krb5.conf examples
EOF
)"
2.4.2. Configuration File Generation
# Create entire config file
sudo tee /etc/krb5.conf > /dev/null << 'EOF'
[libdefaults]
default_realm = INSIDE.DOMUSDIGITALIS.DEV
dns_lookup_realm = false
dns_lookup_kdc = true
[realms]
INSIDE.DOMUSDIGITALIS.DEV = {
kdc = home-dc01.inside.domusdigitalis.dev
admin_server = home-dc01.inside.domusdigitalis.dev
}
[domain_realm]
.inside.domusdigitalis.dev = INSIDE.DOMUSDIGITALIS.DEV
inside.domusdigitalis.dev = INSIDE.DOMUSDIGITALIS.DEV
EOF
2.4.3. gopass Multi-Line Insert
gopass insert ADMINISTRATIO/servers/home-dc01/meta << 'EOF'
hostname: home-dc01
ip: 10.50.1.50
os: Windows Server 2025 Core
domain: inside.domusdigitalis.dev
roles: AD DS, DNS
deployed: 2026-02-09
notes: New forest, replaced old dc-01
EOF
2.4.4. SSH Remote Commands
# Execute multiple commands on remote host
ssh user@server << 'EOF'
cd /var/log
grep -i error syslog | tail -20
df -h
uptime
EOF
2.4.5. SQL Queries
mysql -u admin -p"$PASSWORD" << 'EOF'
SELECT id, username, last_login
FROM users
WHERE last_login < DATE_SUB(NOW(), INTERVAL 90 DAY)
ORDER BY last_login;
EOF
2.4.6. Script Generation
# Generate a script dynamically
cat > /tmp/backup.sh << 'EOF'
#!/bin/bash
set -euo pipefail
BACKUP_DIR="/backup/$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"
for dir in /etc /home /var/log; do
tar -czf "$BACKUP_DIR/$(basename "$dir").tar.gz" "$dir"
done
echo "Backup complete: $BACKUP_DIR"
EOF
chmod +x /tmp/backup.sh
2.5. Herestring (<<<)
For single-line input, use herestring:
# Instead of: echo "hello" | command
# Use: command <<< "hello"
# Example: Base64 encode a string
base64 <<< "secret password"
# Example: Read fields
read -r user host <<< "admin server01"
echo "User: $user, Host: $host"
# Variables are expanded
base64 <<< "User is $USER"
3. Brace Expansion
Brace expansion generates multiple strings from a pattern. It happens BEFORE variable expansion.
3.1. Basic Patterns
# Comma-separated list
echo {a,b,c}
# Output: a b c
echo file{1,2,3}.txt
# Output: file1.txt file2.txt file3.txt
# Nested braces
echo {a,b}{1,2}
# Output: a1 a2 b1 b2
# Empty element
echo file{,-backup}.txt
# Output: file.txt file-backup.txt
3.2. Sequences
# Numeric range
echo {1..5}
# Output: 1 2 3 4 5
# Alphabetic range
echo {a..e}
# Output: a b c d e
# With padding
echo {01..10}
# Output: 01 02 03 04 05 06 07 08 09 10
# With step
echo {0..10..2}
# Output: 0 2 4 6 8 10
# Reverse
echo {5..1}
# Output: 5 4 3 2 1
# Combined with text
echo file{01..05}.log
# Output: file01.log file02.log file03.log file04.log file05.log
3.3. Production Use Cases
3.3.1. Create Multiple Directories
mkdir -p project/{src,docs,tests,config}/{main,backup}
# Creates:
# project/src/main, project/src/backup
# project/docs/main, project/docs/backup
# project/tests/main, project/tests/backup
# project/config/main, project/config/backup
3.3.2. Backup Before Edit
# Copy file to .bak extension
cp /etc/krb5.conf{,.bak}
# Equivalent to: cp /etc/krb5.conf /etc/krb5.conf.bak
# With timestamp
cp /etc/krb5.conf{,.$(date +%Y%m%d)}
# Creates: /etc/krb5.conf.20260209
3.3.3. Rename/Move Files
# Change extension
mv file.{txt,md}
# Equivalent to: mv file.txt file.md
# Change directory
mv /old/path/file.conf{,} /new/path/
# Wait, that doesn't work. Use:
mv /old/path/file.conf /new/path/
3.3.4. Copy Multiple SSH Keys
# Copy several related files
cat ~/.ssh/id_ed25519_{d000,sk_rk_d000,sk_rk_d000_secondary}.pub | wl-copy
3.4. Brace Expansion vs Globs
|
Brace expansion generates strings. It doesn’t check if files exist. Glob patterns match existing files.
|
4. Parameter Expansion
Parameter expansion is the shell’s Swiss Army knife for string manipulation.
4.1. Basic Forms
# Simple expansion
echo $VAR
echo ${VAR} # Preferred - clearer boundaries
# Required: when followed by valid identifier characters
echo ${VAR}able # Correct: "valuable" (if VAR=valu)
echo $VAR_able # Wrong: expands $VAR_able, not $VAR
4.2. Default Values
╔═══════════════════════════════════════════════════════════════════════════╗
║ DEFAULT VALUE OPERATORS ║
╠═══════════════════════════════════════════════════════════════════════════╣
║ ║
║ ${VAR:-default} Use default if VAR is unset OR empty ║
║ ${VAR-default} Use default if VAR is unset (empty is valid) ║
║ ║
║ ${VAR:=default} Set VAR to default if unset OR empty, then use ║
║ ${VAR=default} Set VAR to default if unset (empty is valid) ║
║ ║
║ ${VAR:+value} Use value if VAR is set AND non-empty ║
║ ${VAR+value} Use value if VAR is set (even if empty) ║
║ ║
║ ${VAR:?error} Exit with error if VAR is unset OR empty ║
║ ${VAR?error} Exit with error if VAR is unset ║
║ ║
║ KEY INSIGHT: The colon (:) means "also treat empty as unset" ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════╝
# Use default if unset
echo "${NAME:-anonymous}"
# Set and use default
LOGDIR="${LOGDIR:=/var/log/myapp}"
echo "$LOGDIR"
# Require variable (exit if missing)
: "${CONFIG_FILE:?ERROR: CONFIG_FILE must be set}"
# Conditional value
echo "Debug mode: ${DEBUG:+enabled}"
4.4. Substring Extraction
VAR="hello world"
# From position (0-indexed)
echo ${VAR:6}
# Output: world
# From position with length
echo ${VAR:0:5}
# Output: hello
# Negative offset (from end) - note the space or parentheses
echo ${VAR: -5}
# Output: world
echo ${VAR:(-5)}
# Output: world
# Last 5 characters
echo ${VAR: -5:3}
# Output: wor
4.5. Pattern Removal
╔═══════════════════════════════════════════════════════════════════════════╗
║ PATTERN REMOVAL OPERATORS ║
╠═══════════════════════════════════════════════════════════════════════════╣
║ ║
║ ${VAR#pattern} Remove shortest match from BEGINNING ║
║ ${VAR##pattern} Remove longest match from BEGINNING ║
║ ║
║ ${VAR%pattern} Remove shortest match from END ║
║ ${VAR%%pattern} Remove longest match from END ║
║ ║
║ MEMORY AID: ║
║ # is on the LEFT side of $ on keyboard → removes from LEFT (beginning) ║
║ % is on the RIGHT side of $ on keyboard → removes from RIGHT (end) ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════╝
FILE="/path/to/document.txt"
# Get filename (remove directory path)
echo ${FILE##*/}
# Output: document.txt
# Get directory (remove filename)
echo ${FILE%/*}
# Output: /path/to
# Get extension
echo ${FILE##*.}
# Output: txt
# Remove extension
echo ${FILE%.*}
# Output: /path/to/document
# Remove all extensions (e.g., file.tar.gz)
echo ${FILE%%.*}
# Output: /path/to/document (stops at first dot from right)
# Wait, that's wrong. Let me fix:
FILE2="/path/to/archive.tar.gz"
echo ${FILE2%%.*}
# Output: /path/to/archive
4.6. Pattern Replacement
VAR="hello world world"
# Replace first occurrence
echo ${VAR/world/universe}
# Output: hello universe world
# Replace all occurrences
echo ${VAR//world/universe}
# Output: hello universe universe
# Replace at beginning (like sed ^)
echo ${VAR/#hello/goodbye}
# Output: goodbye world world
# Replace at end (like sed $)
echo ${VAR/%world/universe}
# Output: hello world universe
# Delete (replace with nothing)
echo ${VAR//world}
# Output: hello
4.7. Case Modification (Bash 4.0+)
VAR="Hello World"
# Lowercase first character
echo ${VAR,}
# Output: hello World
# Lowercase all
echo ${VAR,,}
# Output: hello world
# Uppercase first character
echo ${VAR^}
# Output: Hello World
# Uppercase all
echo ${VAR^^}
# Output: HELLO WORLD
# Toggle case
echo ${VAR~~}
# Output: hELLO wORLD
4.8. Indirect Expansion
NAME="greeting"
greeting="Hello, World!"
# Indirect: get value of variable whose name is in NAME
echo ${!NAME}
# Output: Hello, World!
# Useful for dynamic variable names
SERVER="prod"
prod_host="10.50.1.100"
dev_host="10.50.1.200"
echo "Connecting to ${!SERVER@}_host"
# Hmm, that's not quite right. Let me fix:
VAR_NAME="${SERVER}_host"
echo "Connecting to ${!VAR_NAME}"
# Output: Connecting to 10.50.1.100
4.9. Array Operations via Parameter Expansion
arr=(one two three four five)
# All elements
echo "${arr[@]}"
# Output: one two three four five
# Number of elements
echo "${#arr[@]}"
# Output: 5
# Keys/indices
echo "${!arr[@]}"
# Output: 0 1 2 3 4
# Slice: from index 2, take 2 elements
echo "${arr[@]:2:2}"
# Output: three four
5. Arrays
Bash supports indexed (0-based) and associative (key-value) arrays.
5.1. Indexed Arrays
# Declaration methods
arr=(one two three) # Inline
arr[0]="one" # Individual assignment
declare -a arr # Explicit declaration
readarray -t arr < file.txt # From file (one element per line)
# Access elements
echo "${arr[0]}" # First element
echo "${arr[-1]}" # Last element (Bash 4.3+)
echo "${arr[@]}" # All elements (preserves word boundaries)
echo "${arr[*]}" # All elements (as single word)
# Array length
echo "${#arr[@]}" # Number of elements
echo "${#arr[0]}" # Length of first element
# Indices
echo "${!arr[@]}" # All indices
# Append
arr+=("four") # Append element
arr+=(five six) # Append multiple
# Delete
unset 'arr[1]' # Delete element (leaves gap!)
arr=("${arr[@]}") # Reindex to close gaps
# Slice
echo "${arr[@]:1:2}" # Elements 1 and 2 (0-indexed)
# Copy
new_arr=("${arr[@]}")
5.2. Associative Arrays (Bash 4.0+)
# Must declare explicitly
declare -A servers
# Assign
servers[web]="10.50.1.10"
servers[db]="10.50.1.20"
servers[cache]="10.50.1.30"
# Or inline
declare -A servers=(
[web]="10.50.1.10"
[db]="10.50.1.20"
[cache]="10.50.1.30"
)
# Access
echo "${servers[web]}" # Get value
echo "${servers[@]}" # All values
echo "${!servers[@]}" # All keys
# Check if key exists
if [[ -v servers[web] ]]; then
echo "web server defined"
fi
# Iterate
for server in "${!servers[@]}"; do
echo "$server: ${servers[$server]}"
done
# Delete key
unset 'servers[cache]'
5.3. Array Iteration
arr=(one two "three four" five)
# CORRECT: Quoted expansion preserves elements
for item in "${arr[@]}"; do
echo "Item: $item"
done
# Output:
# Item: one
# Item: two
# Item: three four
# Item: five
# WRONG: Unquoted splits on whitespace
for item in ${arr[@]}; do
echo "Item: $item"
done
# Output:
# Item: one
# Item: two
# Item: three
# Item: four
# Item: five
5.4. Array Processing Patterns
# Filter array
arr=(1 2 3 4 5 6 7 8 9 10)
evens=()
for n in "${arr[@]}"; do
(( n % 2 == 0 )) && evens+=("$n")
done
echo "${evens[@]}"
# Output: 2 4 6 8 10
# Map/transform array
files=(file1.txt file2.txt file3.txt)
basenames=()
for f in "${files[@]}"; do
basenames+=("${f%.txt}")
done
echo "${basenames[@]}"
# Output: file1 file2 file3
# Join array to string
IFS=','
echo "${arr[*]}"
# Output: 1,2,3,4,5,6,7,8,9,10
unset IFS
# Split string to array
str="one,two,three,four"
IFS=',' read -ra arr <<< "$str"
echo "${arr[1]}"
# Output: two
6. Command Substitution
Capture command output as a string.
6.1. Syntax
# Modern (preferred)
result=$(command)
# Legacy (avoid - can't nest easily)
result=`command`
# Nested substitution
result=$(echo "Date: $(date +%Y-%m-%d)")
6.2. Use Cases
# Capture command output
files=$(ls -1 *.txt)
count=$(wc -l < file.txt)
hostname=$(hostname -f)
# Use in strings
echo "Current user: $(whoami) on $(hostname)"
# Use in conditionals
if [[ $(whoami) == "root" ]]; then
echo "Running as root"
fi
# Use in arithmetic
file_count=$(ls -1 | wc -l)
(( file_count > 10 )) && echo "Many files"
# Capture exit status AND output
if output=$(command 2>&1); then
echo "Success: $output"
else
echo "Failed: $output"
fi
6.3. Pitfalls
# WRONG: Unquoted loses whitespace
files=$(ls -1)
echo $files # All on one line!
# CORRECT: Quote to preserve
echo "$files" # Preserves newlines
# WRONG: Command substitution in single quotes
echo 'Today is $(date)'
# Output: Today is $(date)
# CORRECT: Use double quotes
echo "Today is $(date)"
# Output: Today is Sun Feb 9 14:32:15 PST 2026
7. Process Substitution
Process substitution provides command output as a file-like object.
7.1. Syntax
# Input: <(command)
# Treats command output as a file you can read
# Output: >(command)
# Treats command input as a file you can write to
7.2. Use Cases
# Compare output of two commands
diff <(ls /etc) <(ls /usr/etc)
# Compare sorted outputs
diff <(sort file1.txt) <(sort file2.txt)
# Feed command output to programs requiring filename
while read -r line; do
echo "Processing: $line"
done < <(find /var/log -name "*.log" -mtime -1)
# Multiple inputs
paste <(cut -f1 file1.txt) <(cut -f1 file2.txt)
# Tee to multiple destinations
command | tee >(grep ERROR > errors.log) >(grep WARN > warnings.log) > full.log
# Write to multiple processes
echo "test" | tee >(md5sum) >(sha256sum) > /dev/null
7.3. Process Substitution vs Pipes
# With pipe, while loop runs in subshell - variables don't persist
count=0
cat file.txt | while read -r line; do
((count++))
done
echo "$count" # Still 0!
# With process substitution, while loop runs in current shell
count=0
while read -r line; do
((count++))
done < <(cat file.txt)
echo "$count" # Correct count!
# Or use lastpipe option (Bash 4.2+)
shopt -s lastpipe
count=0
cat file.txt | while read -r line; do
((count++))
done
echo "$count" # Now works!
8. Clipboard Operations
8.1. Wayland (Modern Linux)
# Copy to clipboard
echo "text" | wl-copy
# Copy file contents
wl-copy < file.txt
cat file.txt | wl-copy
# Copy command output
ls -la | wl-copy
date | wl-copy
# Paste from clipboard
wl-paste
# Paste to file
wl-paste > output.txt
# Copy without newline (for passwords)
printf '%s' "password" | wl-copy
# Copy primary selection (middle-click)
echo "text" | wl-copy --primary
wl-paste --primary
# Watch clipboard and execute on change
wl-paste --watch cat # Prints each new clipboard entry
8.2. X11 (xclip/xsel)
# xclip
echo "text" | xclip -selection clipboard
xclip -selection clipboard -o
# xsel
echo "text" | xsel --clipboard --input
xsel --clipboard --output
# Note: X11 has three selections:
# - PRIMARY (middle-click paste)
# - SECONDARY (rarely used)
# - CLIPBOARD (Ctrl+C/Ctrl+V)
8.3. gopass Integration
# Built-in clipboard copy (auto-clears after 45s)
gopass show -c ADMINISTRATIO/servers/home-dc01/Administrator
# Manual clipboard copy via wl-copy
gopass show -o ADMINISTRATIO/servers/home-dc01/Administrator | wl-copy
# Copy and keep (don't auto-clear)
gopass show -o PATH/TO/SECRET | wl-copy -o
9. Conditionals and Tests
9.1. Test Commands
# Three equivalent syntaxes:
test -f /etc/passwd
[ -f /etc/passwd ]
[[ -f /etc/passwd ]] # Preferred (Bash extension)
9.2. [[ ]] vs [ ]
╔═══════════════════════════════════════════════════════════════════════════╗
║ [[ ]] vs [ ] COMPARISON ║
╠═══════════════════════════════════════════════════════════════════════════╣
║ ║
║ [ ] (test) [[ ]] (Bash keyword) ║
║ ───────────────── ────────────────────── ║
║ POSIX compatible Bash/Zsh/Ksh only ║
║ Word splitting YES Word splitting NO ║
║ Glob expansion YES Glob expansion NO ║
║ Must quote vars Quoting optional* ║
║ -a for AND && for AND ║
║ -o for OR || for OR ║
║ No pattern match Supports =~ regex ║
║ No glob patterns Supports == with globs ║
║ ║
║ *Still good practice to quote in [[ ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════╝
# With [ ], unquoted variable can break
file="my file.txt"
[ -f $file ] # Error: too many arguments
[ -f "$file" ] # Works
# With [[ ]], safe without quotes (but quote anyway)
[[ -f $file ]] # Works
[[ -f "$file" ]] # Also works, preferred
9.3. File Tests
[[ -e "$file" ]] # Exists (any type)
[[ -f "$file" ]] # Regular file
[[ -d "$file" ]] # Directory
[[ -L "$file" ]] # Symbolic link
[[ -r "$file" ]] # Readable
[[ -w "$file" ]] # Writable
[[ -x "$file" ]] # Executable
[[ -s "$file" ]] # Non-empty (size > 0)
[[ -b "$file" ]] # Block device
[[ -c "$file" ]] # Character device
[[ -p "$file" ]] # Named pipe (FIFO)
[[ -S "$file" ]] # Socket
[[ -O "$file" ]] # Owned by current user
[[ -G "$file" ]] # Owned by current group
# File comparisons
[[ "$f1" -nt "$f2" ]] # f1 newer than f2
[[ "$f1" -ot "$f2" ]] # f1 older than f2
[[ "$f1" -ef "$f2" ]] # Same inode (hard link)
9.4. String Tests
[[ -z "$str" ]] # Empty (zero length)
[[ -n "$str" ]] # Non-empty
[[ "$str" ]] # Non-empty (shorthand)
[[ "$a" == "$b" ]] # Equal
[[ "$a" != "$b" ]] # Not equal
[[ "$a" < "$b" ]] # Less than (lexicographic)
[[ "$a" > "$b" ]] # Greater than (lexicographic)
# Pattern matching (glob)
[[ "$str" == *.txt ]] # Ends with .txt
[[ "$str" == [0-9]* ]] # Starts with digit
# Regex matching
[[ "$str" =~ ^[0-9]+$ ]] # All digits
[[ "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]] # Email-ish
9.5. Numeric Tests
# Integer comparison (use (( )) or these operators)
[[ "$a" -eq "$b" ]] # Equal
[[ "$a" -ne "$b" ]] # Not equal
[[ "$a" -lt "$b" ]] # Less than
[[ "$a" -le "$b" ]] # Less than or equal
[[ "$a" -gt "$b" ]] # Greater than
[[ "$a" -ge "$b" ]] # Greater than or equal
# Arithmetic context (preferred for numbers)
(( a == b ))
(( a != b ))
(( a < b ))
(( a <= b ))
(( a > b ))
(( a >= b ))
(( a && b )) # Both non-zero
(( a || b )) # Either non-zero
10. Arithmetic
10.1. Arithmetic Context
# Assignment
(( x = 5 ))
(( y = x + 10 ))
# Increment/decrement
(( x++ ))
(( x-- ))
(( x += 5 ))
# All C operators work
(( x = (5 + 3) * 2 )) # 16
(( x = 10 / 3 )) # 3 (integer division)
(( x = 10 % 3 )) # 1 (modulo)
(( x = 2 ** 8 )) # 256 (exponentiation)
# Comparison (returns 0 for true, 1 for false)
(( 5 > 3 )) && echo "yes"
# Ternary
(( x = (a > b) ? a : b ))
# Bitwise
(( x = 5 & 3 )) # AND: 1
(( x = 5 | 3 )) # OR: 7
(( x = 5 ^ 3 )) # XOR: 6
(( x = ~5 )) # NOT: -6
(( x = 1 << 4 )) # Left shift: 16
(( x = 16 >> 2 )) # Right shift: 4
10.2. $ Arithmetic Expansion
# Returns the value (use when you need the result as a string)
echo "Result: $(( 5 + 3 ))"
result=$(( x * 2 ))
# Variables don't need $ inside
x=5
echo $(( x + 10 )) # 15
echo $(( $x + 10 )) # Also works
# Common patterns
echo "Day $(( $(date +%d) + 1 ))" # Tomorrow's day
11. Pattern Matching (Globs)
11.1. Basic Globs
* # Any string (including empty)
? # Any single character
[abc] # Any one of a, b, c
[a-z] # Any character in range
[!abc] # Any character NOT a, b, c
[^abc] # Same as [!abc]
# Examples
ls *.txt # All .txt files
ls file?.log # file1.log, fileA.log, etc.
ls [Ff]ile.txt # File.txt or file.txt
ls log.[0-9] # log.0 through log.9
11.2. Extended Globs (extglob)
# Enable extended globs
shopt -s extglob
?(pattern) # Zero or one occurrence
*(pattern) # Zero or more occurrences
+(pattern) # One or more occurrences
@(pattern) # Exactly one occurrence
!(pattern) # Anything EXCEPT pattern
# Examples
ls *.@(jpg|png|gif) # Image files
ls !(*.txt) # All files except .txt
ls *.+(tar|gz) # Files ending in tar or gz
rm *.!(keep) # Remove all except .keep files
11.3. Glob Options
# Null glob: no match returns empty (not literal pattern)
shopt -s nullglob
files=(*.nonexistent)
echo "${#files[@]}" # 0 (not 1)
# Fail glob: no match is an error
shopt -s failglob
ls *.nonexistent # Error
# Dot glob: * matches dotfiles
shopt -s dotglob
ls * # Includes .bashrc, etc.
# Glob star: ** matches directories recursively
shopt -s globstar
ls **/*.txt # All .txt files in all subdirs
# No case glob: case-insensitive matching
shopt -s nocaseglob
ls *.TXT # Matches file.txt, FILE.TXT, etc.
12. Job Control
12.1. Background and Foreground
# Run command in background
long_command &
# List jobs
jobs
jobs -l # With PIDs
# Bring to foreground
fg # Most recent job
fg %1 # Job number 1
fg %command # Job starting with "command"
# Suspend foreground job
Ctrl+Z
# Resume in background
bg
bg %1
# Wait for background jobs
wait # All jobs
wait %1 # Specific job
wait $! # Last background process
13. Defensive Scripting Patterns
13.1. Strict Mode
#!/bin/bash
set -euo pipefail
# -e: Exit on error
# -u: Error on unset variables
# -o pipefail: Pipeline fails if any command fails
# Optional additions
set -E # ERR trap inherited by functions
shopt -s inherit_errexit # Command substitutions inherit -e
13.2. Variable Checking
# Require variable to be set
: "${CONFIG_FILE:?ERROR: CONFIG_FILE must be set}"
# Provide default
LOGDIR="${LOGDIR:-/var/log/myapp}"
# Check before use
if [[ -z "${API_KEY:-}" ]]; then
echo "ERROR: API_KEY not set" >&2
exit 1
fi
13.3. Command Existence
# Check if command exists
if ! command -v docker &> /dev/null; then
echo "ERROR: docker not installed" >&2
exit 1
fi
# Or
type jq &> /dev/null || { echo "jq required"; exit 1; }
# Don't use: which, whereis, hash (less reliable)
13.4. File Safety
# Check file before reading
if [[ ! -r "$CONFIG_FILE" ]]; then
echo "Cannot read $CONFIG_FILE" >&2
exit 1
fi
# Check directory before writing
if [[ ! -d "$OUTPUT_DIR" ]]; then
mkdir -p "$OUTPUT_DIR" || exit 1
fi
# Temp file best practice
tmpfile=$(mktemp)
trap 'rm -f "$tmpfile"' EXIT
# Or with directory
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
13.5. Error Handling
# Function for clean error messages
die() {
echo "ERROR: $*" >&2
exit 1
}
# Usage
[[ -f "$file" ]] || die "File not found: $file"
# With cleanup
cleanup() {
rm -f "$tmpfile"
# Other cleanup
}
trap cleanup EXIT
# Catch and handle errors
if ! output=$(command 2>&1); then
echo "Command failed: $output" >&2
exit 1
fi
14. Production Scenarios
14.1. Scenario 1: Backup Script with Safety
#!/bin/bash
set -euo pipefail
# Configuration with defaults
BACKUP_DIR="${BACKUP_DIR:-/backup}"
RETENTION_DAYS="${RETENTION_DAYS:-30}"
declare -a DIRS_TO_BACKUP=(/etc /home /var/log)
# Require root
[[ $EUID -eq 0 ]] || { echo "Must run as root" >&2; exit 1; }
# Verify backup destination
[[ -d "$BACKUP_DIR" ]] || mkdir -p "$BACKUP_DIR"
# Create dated backup directory
DATE=$(date +%Y%m%d_%H%M%S)
CURRENT_BACKUP="$BACKUP_DIR/$DATE"
mkdir -p "$CURRENT_BACKUP"
# Backup each directory
for dir in "${DIRS_TO_BACKUP[@]}"; do
if [[ -d "$dir" ]]; then
name="${dir//\//_}"
tar -czf "$CURRENT_BACKUP/${name}.tar.gz" "$dir"
echo "Backed up: $dir"
else
echo "WARNING: $dir does not exist, skipping" >&2
fi
done
# Cleanup old backups
find "$BACKUP_DIR" -maxdepth 1 -type d -mtime "+$RETENTION_DAYS" -exec rm -rf {} \;
echo "Backup complete: $CURRENT_BACKUP"
14.2. Scenario 2: Log Processing Pipeline
#!/bin/bash
set -euo pipefail
LOGFILE="${1:?Usage: $0 <logfile>}"
[[ -r "$LOGFILE" ]] || { echo "Cannot read $LOGFILE" >&2; exit 1; }
# Count errors by type using associative array
declare -A error_counts
while read -r line; do
if [[ "$line" =~ ERROR:\ ([A-Z_]+) ]]; then
error_type="${BASH_REMATCH[1]}"
(( error_counts[$error_type]++ )) || true
fi
done < "$LOGFILE"
# Output sorted by count
for error_type in "${!error_counts[@]}"; do
echo "${error_counts[$error_type]} $error_type"
done | sort -rn
14.3. Scenario 3: Server Health Check
#!/bin/bash
set -euo pipefail
declare -A servers=(
[web]="10.50.1.10"
[db]="10.50.1.20"
[cache]="10.50.1.30"
)
check_server() {
local name="$1"
local ip="$2"
if ping -c 1 -W 2 "$ip" &> /dev/null; then
echo "[OK] $name ($ip) is reachable"
return 0
else
echo "[FAIL] $name ($ip) is unreachable"
return 1
fi
}
failed=0
for name in "${!servers[@]}"; do
if ! check_server "$name" "${servers[$name]}"; then
((failed++))
fi
done
(( failed == 0 )) || { echo "$failed server(s) unreachable"; exit 1; }
echo "All servers healthy"
14.4. Scenario 4: Configuration Template Expansion
#!/bin/bash
set -euo pipefail
# Variables for template
HOSTNAME="$(hostname -f)"
IP_ADDRESS="$(hostname -I | awk '{print $1}')"
DOMAIN="inside.domusdigitalis.dev"
DATE="$(date +%Y-%m-%d)"
# Template expansion using heredoc with unquoted delimiter
generate_config() {
cat << EOF
# Generated on $DATE for $HOSTNAME
server {
listen 443 ssl;
server_name ${HOSTNAME};
ssl_certificate /etc/ssl/certs/${HOSTNAME}.pem;
ssl_certificate_key /etc/ssl/private/${HOSTNAME}.key;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host \$host;
proxy_set_header X-Real-IP \$remote_addr;
}
}
EOF
}
# Note: \$host and \$remote_addr are escaped to output literal $
# while ${HOSTNAME} is expanded
generate_config > /etc/nginx/conf.d/"${HOSTNAME}".conf
15. Quick Reference
| Pattern | Description |
|---|---|
Heredoc |
|
|
Heredoc, no expansion (literal content) |
|
Heredoc, with variable expansion |
|
Heredoc, strip leading tabs |
|
Herestring (single-line input) |
Parameter Expansion |
|
|
Use default if unset/empty |
|
Set to default if unset/empty |
|
Error if unset/empty |
|
Length of string |
|
Remove shortest match from start |
|
Remove longest match from start |
|
Remove shortest match from end |
|
Remove longest match from end |
|
Replace first occurrence |
|
Replace all occurrences |
|
Lowercase all |
|
Uppercase all |
Brace Expansion |
|
|
Generate: a b c |
|
Generate: 1 2 3 … 10 |
|
Generate: 01 02 03 … 10 (padded) |
|
Generate: 1 3 5 7 9 (step 2) |
|
Copy file to file.bak |
Arrays |
|
|
Create indexed array |
|
First element |
|
All elements (separate words) |
|
Array length |
|
All indices/keys |
|
Append element |
Substitution |
|
|
Command substitution |
|
Process substitution (input) |
|
Process substitution (output) |
Tests |
|
|
File exists and is regular file |
|
Directory exists |
|
String is empty |
|
Strings equal |
|
Regex match |
|
Numeric comparison |
Clipboard (Wayland) |
|
|
Copy to clipboard |
|
Paste from clipboard |