Idempotent Operations

The grep -q || command pattern and its variants — test before acting so operations are safe to repeat. The fundamental building block of reliable scripts and configuration management.

Idempotent Operations

The core principle: test before acting. Run the command 100 times, get the same result as running it once. The guard pattern is test || change — the change only fires when the test fails.

The Fundamental Pattern

grep -q || command — only act if pattern is absent
# Guard: only append if the line doesn't already exist
grep -q 'DIAGRAM_PLANTUML_CLASSPATH' ~/.zshrc || \
  echo 'export DIAGRAM_PLANTUML_CLASSPATH="/usr/share/java/plantuml/plantuml.jar"' >> ~/.zshrc

How it works:

  • grep -q — quiet mode. No output. Exits 0 if found, 1 if not.

  • || — short-circuit OR. Right side runs ONLY when left side fails (pattern absent).

  • Run it twice: first time appends the line. Second time grep -q finds it, exits 0, || skips the append.

Line Insertion with sed

Insert after a specific line, guarded
# Insert PlantUML export after the Ruby gems PATH line
grep -q 'DIAGRAM_PLANTUML_CLASSPATH' ~/.zshrc || \
  sed -i '/export PATH.*gem\/ruby/a\\nexport DIAGRAM_PLANTUML_CLASSPATH="/usr/share/java/plantuml/plantuml.jar"' ~/.zshrc
Insert with awk — more control over placement
# Insert after line matching a pattern, with blank line separation
grep -q 'DIAGRAM_PLANTUML_CLASSPATH' file.conf || \
  awk '/^export PATH.*ruby/{
    print
    print ""
    print "# PlantUML JAR for asciidoctor-diagram"
    print "export DIAGRAM_PLANTUML_CLASSPATH=\"/usr/share/java/plantuml/plantuml.jar\""
    next
  }1' file.conf > /tmp/file.tmp && mv /tmp/file.tmp file.conf

Common Idempotent Guards

Package installation
# Arch Linux
pacman -Q plantuml 2>/dev/null || sudo pacman -S --noconfirm plantuml

# Debian/Ubuntu
dpkg -l plantuml 2>/dev/null | grep -q '^ii' || sudo apt-get install -y plantuml

# Ruby gem
gem list -i asciidoctor-diagram >/dev/null 2>&1 || gem install asciidoctor-diagram
Directory creation
# mkdir -p is already idempotent — but explicit test is clearer
[[ -d /tmp/build ]] || mkdir -p /tmp/build
Symlink creation
# Only stow if the symlink doesn't exist yet
[[ -L ~/.zshrc ]] || stow -t ~ zsh
Service enablement
# Only enable if not already enabled
systemctl is-enabled sshd >/dev/null 2>&1 || sudo systemctl enable sshd
Config file entry
# Append to /etc/hosts only if entry is missing
grep -q '10.50.1.50.*dc01' /etc/hosts || \
  echo '10.50.1.50  dc01.inside.domusdigitalis.dev dc01' | sudo tee -a /etc/hosts
Git remote
# Add remote only if it doesn't exist
git remote get-url backup 2>/dev/null || git remote add backup git@gitea.local:evan/repo.git

Multi-Line Idempotent Block

Append an entire block if a sentinel line is absent
# Guard on a unique line within the block
grep -q '# BEGIN domus-asciidoc-build' ~/.zshrc || cat >> ~/.zshrc << 'EOF'

# BEGIN domus-asciidoc-build
export DIAGRAM_PLANTUML_CLASSPATH="/usr/share/java/plantuml/plantuml.jar"
export ASCIIDOCTOR_DIAGRAM=true
# END domus-asciidoc-build
EOF

The BEGIN/END sentinel pattern: grep checks for the opening marker, the heredoc appends the full block. To later remove it: sed -i '/# BEGIN domus-asciidoc-build/,/# END domus-asciidoc-build/d' ~/.zshrc.

Idempotent Loop Pattern

Process files, skipping already-processed ones
for adoc in data/d001/investigations/**/*.adoc; do
    out="${adoc%.*}.html"
    [[ -f "$out" && "$out" -nt "$adoc" ]] && continue  # skip if output is newer
    build-adoc.sh "$adoc" html --variant catppuccin
done

The -nt (newer than) test: only rebuild when the source has changed. Same principle — test before acting.

Anti-Patterns

Non-idempotent — appends duplicate lines on every run
# BAD: no guard, appends every time
echo 'export FOO=bar' >> ~/.zshrc
Non-idempotent — sed without guard inserts duplicates
# BAD: inserts after every match, every run
sed -i '/pattern/a\new line' file.conf
Fix: wrap with grep -q guard
# GOOD: idempotent
grep -q 'export FOO=bar' ~/.zshrc || echo 'export FOO=bar' >> ~/.zshrc

When NOT to Guard

Not everything needs idempotency. Commands that are naturally idempotent:

  • mkdir -p — creates if absent, no-ops if present

  • cp -u — copies only if source is newer

  • ln -sf — force-creates, replacing existing

  • chmod 600 file — sets permissions regardless of current state

  • git config --global user.name "Evan" — overwrites, same result each time

The rule: if the command’s effect is the same whether run once or many times, no guard is needed. If it accumulates (append, insert, add), guard it.

See Also