Advanced Bash Patterns

Patterns that separate intermediate from advanced shell users.

Progression Path

Level Indicators Status

Intermediate

Terminal-first, pipelines, find+grep combos, heredocs

[x] You Are Here

Advanced

Process substitution, named pipes, signal handling

[ ] Training

Expert

Teaches others, writes portable scripts, designs workflows

[ ] Future

Module 1: Process Substitution

The Concept

Process substitution creates a temporary file descriptor containing command output. The receiving command sees a path like /dev/fd/63.

Syntax:

<(command)   # Output substitution (most common)
>(command)   # Input substitution (rare)

Basic Patterns

# Compare two command outputs
diff <(ls ~/dir1) <(ls ~/dir2)

# Compare sorted vs original
diff <(sort file.txt) file.txt

# Compare remote vs local config
diff <(ssh server cat /etc/ssh/sshd_config) /etc/ssh/sshd_config

# Feed multiple inputs to paste
paste <(cut -d: -f1 /etc/passwd) <(cut -d: -f7 /etc/passwd)

# Compare two git branches
diff <(git show main:file.txt) <(git show feature:file.txt)

Infrastructure Drills

Type these by hand. Do not copy-paste.

Drill 1: Compare antora.yml attributes across repos

diff <(grep -E "^    [a-z]" ~/atelier/_bibliotheca/domus-infra-ops/docs/asciidoc/antora.yml | sort) \
     <(grep -E "^    [a-z]" ~/atelier/_bibliotheca/domus-captures/docs/asciidoc/antora.yml | sort)

Drill 2: Compare DNS records from two servers

diff <(dig @10.50.1.90 inside.domusdigitalis.dev AXFR | sort) \
     <(dig @10.50.1.91 inside.domusdigitalis.dev AXFR | sort)

Drill 3: Compare running processes across hosts

diff <(ssh kvm-01 "ps aux --no-headers | awk '{print \$11}' | sort -u") \
     <(ssh kvm-02 "ps aux --no-headers | awk '{print \$11}' | sort -u")

Drill 4: Compare package lists

diff <(pacman -Qq | sort) <(cat ~/.config/packages/base.txt | sort)

Why Not Temp Files?

Temp Files Process Substitution

Must create, clean up

Automatic, ephemeral

Disk I/O

Memory only (via /dev/fd)

Race conditions possible

Atomic execution

Clutters filesystem

No artifacts

Input Substitution (Advanced)

# Write to multiple files simultaneously
echo "log message" | tee >(logger -t myapp) >(cat >> /var/log/myapp.log)

# Send output to both file and command
command | tee >(wc -l > /tmp/linecount) > output.txt

Completion Criteria

[ ] Write diff <() without reference [ ] Use for comparing remote vs local files [ ] Use for comparing git branches [ ] Use for comparing sorted/filtered versions

Module 2: Named Pipes (FIFOs)

The Concept

A named pipe (FIFO) is a file that acts as a pipe between processes. Data written to one end can be read from the other.

Key properties:

  • Persists in filesystem (unlike anonymous pipes)

  • Blocks until both reader and writer connect

  • First-In-First-Out data order

  • Zero disk I/O (kernel buffer only)

Creation and Basic Use

# Create a named pipe
mkfifo /tmp/mypipe

# Terminal 1: Reader (blocks until writer connects)
cat /tmp/mypipe

# Terminal 2: Writer
echo "hello from another process" > /tmp/mypipe

# Clean up
rm /tmp/mypipe

Practical Patterns

Pattern 1: Log aggregation

mkfifo /tmp/logs

# Reader: process all logs
while read line; do
    echo "[$(date +%H:%M:%S)] $line"
done < /tmp/logs &

# Writers: multiple processes send logs
echo "app1: started" > /tmp/logs
echo "app2: connected" > /tmp/logs

Pattern 2: Progress monitoring

mkfifo /tmp/progress

# Reader: display progress
while read pct; do
    printf "\rProgress: %3d%%" "$pct"
done < /tmp/progress &

# Writer: long-running task
for i in {1..100}; do
    sleep 0.1
    echo "$i" > /tmp/progress
done
echo  # newline after progress bar

Pattern 3: Inter-process communication

mkfifo /tmp/request /tmp/response

# Server process
while true; do
    read cmd < /tmp/request
    case "$cmd" in
        status) echo "OK" > /tmp/response ;;
        stop)   echo "Stopping" > /tmp/response; break ;;
        *)      echo "Unknown: $cmd" > /tmp/response ;;
    esac
done &

# Client process
echo "status" > /tmp/request
read reply < /tmp/response
echo "Server said: $reply"

Infrastructure Drill

Drill: Real-time log filter

mkfifo /tmp/syslog_filter

# Reader: show only SSH-related logs
grep --line-buffered "sshd" < /tmp/syslog_filter &

# Writer: tail syslog to pipe
tail -f /var/log/auth.log > /tmp/syslog_filter

FIFO vs Anonymous Pipe

Aspect Anonymous Pipe (|) Named Pipe (FIFO)

Scope

Same shell session

Any process, any time

Lifetime

Command duration

Until explicitly removed

Use case

Linear pipelines

IPC, daemons, parallel processing

Completion Criteria

[ ] Create and use FIFO with mkfifo [ ] Implement reader/writer pattern [ ] Use for log aggregation [ ] Use for progress monitoring

Module 3: Signal Handling

The Concept

Signals are software interrupts sent to processes. Scripts can trap signals and execute custom handlers.

Common Signals

Signal Number Description

SIGHUP

1

Hangup (terminal closed)

SIGINT

2

Interrupt (Ctrl+C)

SIGQUIT

3

Quit (Ctrl+\)

SIGTERM

15

Termination request

SIGKILL

9

Force kill (cannot trap)

SIGUSR1

10

User-defined 1

SIGUSR2

12

User-defined 2

Basic Trap Syntax

# Trap single signal
trap 'echo "Interrupted!"' SIGINT

# Trap multiple signals
trap 'cleanup' SIGINT SIGTERM SIGHUP

# Trap EXIT (always runs on script exit)
trap 'cleanup' EXIT

# Reset trap to default
trap - SIGINT

# Ignore signal
trap '' SIGINT

Cleanup Pattern (Most Important)

#!/bin/bash

TEMPDIR=$(mktemp -d)
LOCKFILE="/tmp/myscript.lock"

cleanup() {
    echo "Cleaning up..."
    rm -rf "$TEMPDIR"
    rm -f "$LOCKFILE"
    exit "${1:-0}"
}

# Trap all exit scenarios
trap cleanup EXIT
trap 'cleanup 130' SIGINT
trap 'cleanup 143' SIGTERM

# Create lock
echo $$ > "$LOCKFILE"

# Main script work
echo "Working in $TEMPDIR"
sleep 60

# Normal exit triggers cleanup via EXIT trap

Progress with Graceful Shutdown

#!/bin/bash

RUNNING=true

handle_signal() {
    echo -e "\nReceived signal, finishing current task..."
    RUNNING=false
}

trap handle_signal SIGINT SIGTERM

for i in {1..100}; do
    $RUNNING || break
    echo "Processing item $i"
    sleep 1
done

echo "Graceful shutdown complete"

Infrastructure Drill

Drill: Service wrapper with PID file

#!/bin/bash
# service-wrapper.sh

PIDFILE="/var/run/myservice.pid"
LOGFILE="/var/log/myservice.log"

cleanup() {
    echo "[$(date)] Service stopping" >> "$LOGFILE"
    rm -f "$PIDFILE"
    exit 0
}

trap cleanup SIGINT SIGTERM EXIT

# Write PID
echo $$ > "$PIDFILE"
echo "[$(date)] Service started (PID $$)" >> "$LOGFILE"

# Main loop
while true; do
    echo "[$(date)] Heartbeat" >> "$LOGFILE"
    sleep 30
done

Reload Configuration Pattern

#!/bin/bash

CONFIG_FILE="/etc/myapp/config"
CONFIG_VAL=""

load_config() {
    echo "Loading config from $CONFIG_FILE"
    CONFIG_VAL=$(cat "$CONFIG_FILE" 2>/dev/null || echo "default")
}

trap load_config SIGHUP

load_config  # Initial load

while true; do
    echo "Running with config: $CONFIG_VAL"
    sleep 5
done

# Usage: kill -HUP $PID to reload config

Completion Criteria

[ ] Write cleanup trap for temp files [ ] Implement graceful shutdown [ ] Use SIGHUP for config reload [ ] Understand which signals can/cannot be trapped

Module 4: Subshells and Command Grouping

Subshells ( )

Commands in parentheses run in a subshell (child process). Variable changes do not affect parent.

# Subshell isolation
VAR="original"
( VAR="changed"; echo "Inside: $VAR" )
echo "Outside: $VAR"
# Output:
# Inside: changed
# Outside: original

# Temporary directory change
( cd /tmp && pwd )  # prints /tmp
pwd                 # still in original directory

# Parallel execution
( sleep 5; echo "Task 1 done" ) &
( sleep 3; echo "Task 2 done" ) &
wait

Command Grouping { }

Commands in braces run in current shell. Variable changes persist.

# Grouped output
{ echo "line1"; echo "line2"; } > output.txt

# Grouped redirection
{
    echo "=== Header ==="
    cat data.txt
    echo "=== Footer ==="
} | tee report.txt

# Variable persistence
VAR="original"
{ VAR="changed"; }
echo "$VAR"  # prints "changed"

Practical Patterns

Pattern: Atomic file writes

{
    echo "key1=value1"
    echo "key2=value2"
    echo "key3=value3"
} > /tmp/config.tmp && mv /tmp/config.tmp /etc/myapp.conf

Pattern: Timed subshell

timeout 30 bash -c '( while true; do echo "working..."; sleep 1; done )'

Pattern: Isolated environment

(
    export PATH="/custom/bin:$PATH"
    export MY_VAR="isolated"
    ./special-script.sh
)
# PATH and MY_VAR unchanged here

Completion Criteria

[ ] Understand subshell vs grouping difference [ ] Use subshells for isolation [ ] Use grouping for combined output [ ] Use for temporary environment changes

Module 5: Coprocesses

The Concept

Coprocesses are background processes with bidirectional pipes. You can write to and read from them.

# Start coprocess
coproc BC { bc -l; }

# Write to coprocess
echo "scale=10; 22/7" >&${BC[1]}

# Read from coprocess
read result <&${BC[0]}
echo "Pi approx: $result"

# Close and cleanup
exec {BC[1]}>&-
wait $BC_PID

Practical Example

# Interactive calculator session
coproc CALC { bc -l; }

calc() {
    echo "$1" >&${CALC[1]}
    read result <&${CALC[0]}
    echo "$result"
}

echo "2+2 = $(calc '2+2')"
echo "sqrt(2) = $(calc 'sqrt(2)')"
echo "e^1 = $(calc 'e(1)')"

exec {CALC[1]}>&-

Infrastructure Application

# Persistent SSH session for multiple commands
coproc SSH { ssh -T server; }

ssh_cmd() {
    echo "$1" >&${SSH[1]}
    # Note: reading requires careful handling of output
}

ssh_cmd "hostname"
ssh_cmd "uptime"
ssh_cmd "exit"
wait $SSH_PID

Daily Practice

Add one pattern to your workflow each day:

| Day | Pattern | Practice | |-----|---------|----------| | 1 | diff <() <() | Compare two command outputs | | 2 | Process substitution | Use with jq/awk | | 3 | mkfifo | Create and use named pipe | | 4 | trap cleanup EXIT | Add to any script you write | | 5 | trap SIGINT SIGTERM | Graceful shutdown | | 6 | Subshells | Isolate environment | | 7 | Grouping | Combined output |

Quick Reference

# Process substitution
diff <(cmd1) <(cmd2)
paste <(cmd1) <(cmd2)
cmd <(generate_input)

# Named pipes
mkfifo /tmp/pipe
cmd1 > /tmp/pipe &
cmd2 < /tmp/pipe

# Signal handling
trap 'cleanup' EXIT SIGINT SIGTERM
trap 'reload_config' SIGHUP
trap '' SIGINT  # ignore
trap - SIGINT   # reset

# Subshells (isolated)
( cd /tmp && cmd )
( export VAR=x; cmd )

# Grouping (current shell)
{ cmd1; cmd2; } > file
{ cmd1; cmd2; } | cmd3

# Coprocess
coproc NAME { cmd; }
echo "input" >&${NAME[1]}
read output <&${NAME[0]}