Bash Tools Mastery

Master the essential Unix tools that transform raw data into actionable intelligence. This curriculum builds the muscle memory for find, grep, sed, awk, xargs, and advanced patterns.

The Unix Philosophy

Write programs that do one thing and do it well. Write programs to work together.
— Doug McIlroy

Every tool is a filter. Data flows through pipelines. Composition creates power.

Why These Tools?

Tool Purpose Daily Use

find

Locate files by name, type, time, size

"Where’s that config file I edited yesterday?"

grep

Search file contents

"Which files contain this IP address?"

sed

Transform text streams

"Replace all occurrences of X with Y"

awk

Column-based processing

"Extract field 3 from CSV, sum column 4"

xargs

Build commands from input

"Delete all .tmp files older than 7 days"

Process Sub

Treat commands as files

"Diff outputs of two commands"

Curriculum

Module Description Level

find

File discovery, -exec, -type, time-based filtering

Core

grep

Pattern matching, BRE/ERE/PCRE, context, file filtering

Core

sed

Stream editing, substitution, addresses, hold buffer

Core

awk

Field processing, patterns, BEGIN/END, arrays

Intermediate

xargs

Command composition, parallel execution, batching

Intermediate

Process Substitution

<(), >(), FIFOs, subshells, command grouping

Advanced

Signals & Jobs

Process control, traps, background jobs

Advanced

The Mastery Path

Level 1: Survival (Week 1-2)

  • find files by name and type

  • grep for strings in files

  • sed for simple substitutions

  • Basic pipes: command | grep | sort | uniq

Level 2: Competence (Week 3-4)

  • find with -exec and time filters

  • grep with regex and context

  • sed with addresses and multiple commands

  • awk for field extraction

  • xargs for batch operations

Level 3: Proficiency (Month 2)

  • Complex find predicates

  • PCRE in grep (-P)

  • sed hold buffer

  • awk arrays and functions

  • Process substitution

Level 4: Mastery (Month 3+)

  • Compose tools for novel problems

  • Write one-liners from memory

  • Know when to switch to Python

  • Teach others

Quick Reference

Find

# Find by name
find . -name "*.adoc"

# Find by type (f=file, d=dir, l=link)
find . -type f -name "*.log"

# Find modified in last 24h
find . -mtime -1

# Find and execute
find . -name "*.tmp" -exec rm {} \;

# Find and xargs (faster)
find . -name "*.tmp" -print0 | xargs -0 rm

Grep

# Basic search
grep "pattern" file

# Recursive
grep -r "pattern" dir/

# Extended regex
grep -E "error|warning" file

# PCRE (lookahead, etc.)
grep -P "(?<=user=)\w+" file

# Files only
grep -l "pattern" *.txt

# Count matches
grep -c "pattern" file

# Context
grep -A2 -B2 "pattern" file

Sed

# Substitute
sed 's/old/new/' file

# Global substitute
sed 's/old/new/g' file

# In-place
sed -i 's/old/new/g' file

# Line range
sed -n '10,20p' file

# Delete lines
sed '/pattern/d' file

# Multiple commands
sed -e 's/a/A/' -e 's/b/B/' file

Awk

# Print column
awk '{print $2}' file

# Field separator
awk -F: '{print $1}' /etc/passwd

# Pattern match
awk '/error/ {print}' file

# Sum column
awk '{sum+=$3} END {print sum}' file

# Line range
awk 'NR>=10 && NR<=20' file

# Conditional
awk '$3 > 100 {print $1, $3}' file

Xargs

# Basic
cat files.txt | xargs rm

# Null-delimited (safe)
find . -name "*.tmp" -print0 | xargs -0 rm

# Parallel
cat urls.txt | xargs -P4 -I{} curl {}

# Batching
cat items.txt | xargs -n10 echo

The Practice Method

Daily Drills

  1. Morning warm-up: Find something in your home directory

  2. Log review: grep through yesterday’s logs

  3. Data extraction: awk a field from system output

  4. Batch operation: xargs something

The "Harder Way" Rule

When you reach for the easy command, ask: "What’s the awk/sed way?"

Easy Way Hard Way (Practice This)

head -20 file

awk 'NR⇐20' file

grep -c pattern file

awk '/pattern/{c++} END{print c}' file

cut -d: -f1 /etc/passwd

awk -F: '{print $1}' /etc/passwd

wc -l file

awk 'END{print NR}' file

Composition Patterns

Pipeline Pattern

# Generate → Filter → Transform → Output
find . -name "*.log" |
  xargs grep -l "ERROR" |
  xargs awk '/ERROR/{print FILENAME": "$0}' |
  sort -u

Diff Pattern

# Compare command outputs
diff <(cmd1) <(cmd2)

# Compare sorted outputs
diff <(sort file1) <(sort file2)

Tee Pattern

# Process while saving
cmd | tee output.log | grep "ERROR"

Parallel Pattern

# Process multiple files in parallel
find . -name "*.json" -print0 |
  xargs -0 -P4 -I{} jq '.field' {}

Related Resources