sed Deep Dive

The Golden Rule

Preview before apply. Always.

# Preview (shows what WOULD change)
sed -n 's/old/new/gp' file

# Apply (actually changes the file)
sed -i 's/old/new/g' file

This is non-negotiable. One wrong sed -i on /etc/fstab or /etc/ssh/sshd_config and you’re in recovery mode.

Understanding sed Architecture

╔══════════════════════════════════════════════════════════════════════════════╗
║                           SED PROCESSING MODEL                               ║
╠══════════════════════════════════════════════════════════════════════════════╣
║                                                                              ║
║   INPUT FILE                                                                 ║
║       │                                                                      ║
║       ▼                                                                      ║
║   ┌─────────────────────────────────────────────────────────────────────┐   ║
║   │                      PATTERN SPACE                                  │   ║
║   │   (Current line being processed - where commands operate)           │   ║
║   └─────────────────────────────────────────────────────────────────────┘   ║
║       │                           ▲                                          ║
║       │ (exchange: x, h, H, g, G) │                                          ║
║       ▼                           │                                          ║
║   ┌─────────────────────────────────────────────────────────────────────┐   ║
║   │                       HOLD SPACE                                    │   ║
║   │   (Auxiliary buffer for multi-line operations)                      │   ║
║   └─────────────────────────────────────────────────────────────────────┘   ║
║       │                                                                      ║
║       ▼                                                                      ║
║   OUTPUT (stdout or file with -i)                                            ║
║                                                                              ║
║   Processing cycle per line:                                                 ║
║   1. Read line into pattern space                                            ║
║   2. Execute all commands                                                    ║
║   3. Print pattern space (unless -n)                                         ║
║   4. Clear pattern space, repeat                                             ║
║                                                                              ║
╚══════════════════════════════════════════════════════════════════════════════╝

Command Reference

Substitution (s)

The workhorse command.

sed 's/pattern/replacement/flags'

Flags

Flag Description Example

g

Global - all occurrences on line

s/a/b/g

p

Print - output modified line

s/a/b/gp

i / I

Case-insensitive match

s/error/ERROR/gi

1, 2, n

Replace only nth occurrence

s/a/b/2

w file

Write matched lines to file

s/pattern/new/w matches.txt

Delete (d)

# Delete lines matching pattern
sed '/pattern/d' file

# Delete line 5
sed '5d' file

# Delete lines 5-10
sed '5,10d' file

# Delete from pattern to end
sed '/START/,$d' file

# Delete empty lines
sed '/^$/d' file

# Delete lines starting with #
sed '/^#/d' file

# Delete blank and comment lines
sed '/^$/d; /^#/d' file

Insert (i) and Append (a)

# Insert BEFORE line 5
sed '5i\New line inserted before' file

# Append AFTER line 5
sed '5a\New line appended after' file

# Insert before lines matching pattern
sed '/pattern/i\Inserted before match' file

# Append after lines matching pattern
sed '/pattern/a\Appended after match' file

# Multi-line insert (use \ for continuation)
sed '1i\
# Configuration file\
# Generated: 2026-02-09\
# DO NOT EDIT MANUALLY' file

Change (c)

Replace entire line:

# Replace line 5 entirely
sed '5c\This replaces line 5 completely' file

# Replace lines matching pattern
sed '/old_config/c\new_config = enabled' file

Print (p)

# Print matching lines (with -n to suppress default output)
sed -n '/ERROR/p' file

# Print line range
sed -n '10,20p' file

# Print first 10 lines (like head)
sed -n '1,10p' file

# Print last line
sed -n '$p' file

# Print every other line (odd lines)
sed -n '1~2p' file

# Print every 3rd line starting from line 2
sed -n '2~3p' file

Quit (q)

# Print first 10 lines and quit (faster than head for huge files)
sed '10q' file

# Quit after finding pattern
sed '/FOUND/q' file

# Print up to and including pattern, then quit
sed -n '/pattern/{p;q}' file

Transform (y)

Character-by-character translation (like tr):

# Lowercase to uppercase
sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' file

# ROT13
sed 'y/abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM/' file

Address Ranges

Control WHICH lines commands operate on.

Line Numbers

# Single line
sed '5s/old/new/' file

# Range
sed '5,10s/old/new/' file

# Line 5 to end of file
sed '5,$s/old/new/' file

# First line only
sed '1s/old/new/' file

# Last line only
sed '$s/old/new/' file

Step Intervals

# Every line starting from line 1, step 2 (odd lines)
sed '1~2s/old/new/' file

# Every 5th line starting from line 0
sed '0~5s/old/new/' file

Pattern Addresses

# Lines containing pattern
sed '/ERROR/s/old/new/' file

# Lines NOT containing pattern
sed '/ERROR/!s/old/new/' file

# From first match to second match
sed '/START/,/END/s/old/new/' file

# From line 10 to pattern
sed '10,/STOP/s/old/new/' file

# From pattern to end of file
sed '/BEGIN/,$s/old/new/' file

Combining Addresses

# Lines 10-20 containing "error"
sed '10,20{/error/s/old/new/}' file

# All lines EXCEPT 1-5
sed '1,5!s/old/new/' file

# Multiple ranges
sed -e '1,10s/old/new/' -e '20,30s/old/new/' file

The Hold Buffer: Multi-Line Operations

The hold space is sed’s secret weapon for complex transformations.

Hold Buffer Commands

Command Action

h

Copy pattern space to hold space (overwrites)

H

Append pattern space to hold space (with newline)

g

Copy hold space to pattern space (overwrites)

G

Append hold space to pattern space (with newline)

x

Exchange pattern space and hold space

Reverse Lines (tac implementation)

# Reverse order of lines
sed '1!G;h;$!d' file

# How it works:
# 1!G  - For all lines except first, append hold to pattern
# h    - Copy pattern to hold
# $!d  - Delete pattern unless last line
# Result: Lines accumulate in reverse order

Join Every Two Lines

# Join pairs of lines with space
sed 'N;s/\n/ /' file

# N = append next line to pattern space
# s/\n/ / = replace newline with space

Print Paragraph Containing Pattern

# Print paragraph (separated by blank lines) containing "ERROR"
sed -n '/^$/,/^$/H; /ERROR/{x;p;}' file

Delete Duplicate Consecutive Lines (uniq)

sed '$!N; /^\(.*\)\n\1$/!P; D' file

Regex Mastery

Basic Regex (BRE) - Default

.     Any single character
*     Zero or more of previous
^     Start of line
$     End of line
[abc] Character class
[^abc] Negated character class
\{n\} Exactly n occurrences
\{n,\} n or more
\{n,m\} Between n and m
\(\)  Capture group (backreference with \1, \2...)

Extended Regex (ERE) - With -E

sed -E 's/pattern/replacement/' file

+     One or more of previous
?     Zero or one of previous
|     Alternation (or)
()    Grouping (no escaping needed)
{n}   Exactly n (no escaping needed)

Escaping Reference

Character BRE ERE (-E)

Literal .

\.

\.

Literal *

\*

\*

Literal [

\[

\[

Literal $

\$

\$

Literal \

\\

\\

One or more

\+

+

Zero or one

\?

?

Alternation

|

|

Grouping

\(\)

()

Backreferences

# Swap two words
echo "hello world" | sed 's/\(\w*\) \(\w*\)/\2 \1/'
# Output: world hello

# Duplicate matched text
echo "abc" | sed 's/\(.*\)/\1\1/'
# Output: abcabc

# Reference entire match with &
echo "hello" | sed 's/.*/[&]/'
# Output: [hello]

# Extract parts
echo "user@domain.com" | sed 's/\(.*\)@\(.*\)/User: \1, Domain: \2/'
# Output: User: user, Domain: domain.com

Production Scenarios

Configuration File Management

Update Kerberos DC Reference

# Preview
sed -n 's/dc-01\.inside\.domusdigitalis\.dev/home-dc01.inside.domusdigitalis.dev/gp' /etc/krb5.conf

# Apply
sudo sed -i 's/dc-01\.inside\.domusdigitalis\.dev/home-dc01.inside.domusdigitalis.dev/g' /etc/krb5.conf

SSH Hardening

# Disable password authentication
sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config

# Disable root login
sudo sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config

# Change SSH port
sudo sed -i 's/^#\?Port.*/Port 22022/' /etc/ssh/sshd_config

# Enable key-only auth (multiple changes)
sudo sed -i '
    s/^#\?PasswordAuthentication.*/PasswordAuthentication no/
    s/^#\?PubkeyAuthentication.*/PubkeyAuthentication yes/
    s/^#\?ChallengeResponseAuthentication.*/ChallengeResponseAuthentication no/
' /etc/ssh/sshd_config

Update IP Addresses Across Configs

# Find all files containing old IP
grep -rl "10.50.1.50" /etc/ 2>/dev/null

# Preview changes in each file
for f in $(grep -rl "10.50.1.50" /etc/ 2>/dev/null); do
    echo "=== $f ==="
    sed -n 's/10\.50\.1\.50/10.50.1.51/gp' "$f"
done

# Apply (with backup)
for f in $(grep -rl "10.50.1.50" /etc/ 2>/dev/null); do
    sudo sed -i.bak 's/10\.50\.1\.50/10.50.1.51/g' "$f"
done

Log Processing

Extract Specific Fields

# Extract IP addresses from Apache log
sed -n 's/^\([0-9.]*\) .*/\1/p' /var/log/apache2/access.log

# Extract timestamps
sed -n 's/.*\[\([^]]*\)\].*/\1/p' /var/log/apache2/access.log

# Extract URLs (between quotes after method)
sed -n 's/.*"\(GET\|POST\) \([^ ]*\).*/\2/p' /var/log/apache2/access.log

Filter Log Levels

# Show only ERROR and CRITICAL lines
sed -n '/\(ERROR\|CRITICAL\)/p' /var/log/application.log

# Remove DEBUG lines for cleaner output
sed '/DEBUG/d' /var/log/application.log

# Show context: 2 lines before and after ERROR
sed -n '/ERROR/{x;p;x;p;n;p;n;p}' /var/log/application.log

Anonymize Logs

# Replace IP addresses with [REDACTED]
sed 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/[REDACTED]/g' access.log

# Replace email addresses
sed 's/[A-Za-z0-9._%+-]*@[A-Za-z0-9.-]*\.[A-Za-z]\{2,\}/[EMAIL]/g' file

# Replace credit card numbers (basic pattern)
sed 's/[0-9]\{4\}[- ]\?[0-9]\{4\}[- ]\?[0-9]\{4\}[- ]\?[0-9]\{4\}/[CC-REDACTED]/g' file

Certificate and Key Operations

Extract Certificate from Combined File

# Extract just the certificate
sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' combined.pem > cert.pem

# Extract just the private key
sed -n '/-----BEGIN.*PRIVATE KEY-----/,/-----END.*PRIVATE KEY-----/p' combined.pem > key.pem

# Extract CA chain (all certs except first)
sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/{
    1,/-----END CERTIFICATE-----/d
    p
}' fullchain.pem > chain.pem

Format Certificate for One-Line Storage

# Convert newlines to \n for JSON/database storage
sed ':a;N;$!ba;s/\n/\\n/g' cert.pem

# Reverse: restore newlines
echo "$ONE_LINE_CERT" | sed 's/\\n/\n/g'

JSON/YAML Manipulation

Simple JSON Value Update

# Update a simple value (use jq for complex JSON)
sed 's/"version": "[^"]*"/"version": "2.0.0"/' package.json

# Update port number
sed 's/"port": [0-9]*/"port": 8443/' config.json

YAML Key Update

# Update simple YAML value
sed 's/^\(  port:\).*/\1 8443/' config.yaml

# Comment out a YAML section
sed '/^database:/,/^[a-z]/s/^/#/' config.yaml

Systemd and Service Files

Modify Unit File

# Change ExecStart
sudo sed -i 's|^ExecStart=.*|ExecStart=/usr/local/bin/myapp --config /etc/myapp/prod.conf|' /etc/systemd/system/myapp.service

# Add After= dependency
sudo sed -i '/^\[Unit\]/a After=network-online.target' /etc/systemd/system/myapp.service

# Change restart policy
sudo sed -i 's/^Restart=.*/Restart=always/' /etc/systemd/system/myapp.service

Bulk File Operations

Rename Variables Across Codebase

# Find files, preview changes
find . -name "*.py" -exec grep -l "old_function" {} \; | while read f; do
    echo "=== $f ==="
    sed -n 's/old_function/new_function/gp' "$f"
done

# Apply changes
find . -name "*.py" -exec sed -i 's/old_function/new_function/g' {} \;

Add Copyright Header to All Source Files

HEADER='# Copyright 2026 Domus Digitalis\n# SPDX-License-Identifier: MIT\n'

find . -name "*.py" -exec sed -i "1i\\
# Copyright 2026 Domus Digitalis\\
# SPDX-License-Identifier: MIT\\
" {} \;

Advanced Techniques

Branching and Labels

# Syntax:
# :label    - Define label
# b label   - Branch to label
# t label   - Branch if substitution succeeded
# T label   - Branch if substitution failed (GNU extension)

# Loop until no more changes
sed ':loop; s/  / /g; t loop' file
# Replaces multiple spaces with single space

# Conditional processing
sed '/pattern/{
    s/old/new/
    t done
    s/fallback/replacement/
    :done
}' file

Multi-Line Pattern Matching

# N command: Append next line to pattern space

# Match pattern spanning two lines
sed 'N; s/line1\nline2/replacement/' file

# Join continuation lines (ending with \)
sed ':a; /\\$/{N; s/\\\n//; ta}' file

# Delete multi-line block
sed '/START/,/END/d' file

In-Place Editing with Atomic Safety

# Problem: sed -i can corrupt files on crash

# Safer: Use temporary file
sed 's/old/new/g' file > file.tmp && mv file.tmp file

# With backup
cp file file.bak && sed 's/old/new/g' file.bak > file

# Atomic with sponge (from moreutils)
sed 's/old/new/g' file | sponge file

Script Files

Create reusable sed scripts:

hardening.sed
# SSH Hardening Script
# Usage: sed -f hardening.sed /etc/ssh/sshd_config

# Disable password auth
s/^#\?PasswordAuthentication.*/PasswordAuthentication no/

# Disable root login
s/^#\?PermitRootLogin.*/PermitRootLogin no/

# Use strong ciphers only
/^#\?Ciphers/c\
Ciphers aes256-gcm@openssh.com,aes128-gcm@openssh.com

# Set idle timeout
/^#\?ClientAliveInterval/c\
ClientAliveInterval 300
# Apply script
sudo sed -f hardening.sed -i.bak /etc/ssh/sshd_config

Debugging sed

Show What’s Happening

# Mark all substitutions with brackets
sed 's/pattern/[MATCH: &]/g' file

# Print line numbers of matches
sed -n '/pattern/{=;p}' file

# Verbose: Print before and after each command (GNU sed)
sed --debug 's/old/new/g' file 2>&1 | head -50

Common Pitfalls

1. Greedy Matching

# Problem: .* is greedy
echo "start middle end" | sed 's/start.*end/REPLACED/'
# Matches: "start middle end" (entire thing)

# Solution: Use negated character class
echo "<tag>content</tag>" | sed 's/<[^>]*>/[TAG]/g'
# Output: [TAG]content[TAG]

2. Missing -i Backup

# Dangerous
sed -i 's/old/new/g' /etc/important.conf

# Safe
sed -i.bak 's/old/new/g' /etc/important.conf

3. Unescaped Special Characters

# WRONG
sed 's/192.168.1.1/10.0.0.1/g'  # Matches 192x168y1z1

# RIGHT
sed 's/192\.168\.1\.1/10.0.0.1/g'

4. Delimiter Conflicts

# WRONG (path contains /)
sed 's//old/path//new/path/g'

# RIGHT (use different delimiter)
sed 's|/old/path|/new/path|g'

# WRONG (pattern contains / like vim search patterns)
sed 's/{pattern}/\\{pattern}/g' file     # Error: unknown option to 's'

# RIGHT (use | when pattern has /)
sed 's|{pattern}|\\{pattern}|g' file     # Works!

# REAL-WORLD: Escape AsciiDoc attribute placeholders
# Patterns like /{pattern}, ?{pattern}, `f{char}` contain /
sed -i 's|{char}|\\{char}|g' shortcuts.adoc
sed -i 's|{motion}|\\{motion}|g' shortcuts.adoc

Performance Considerations

Large Files

# For huge files, quit early if possible
sed '/pattern/{p;q}' hugefile.log

# Process only first N lines
sed '1000q' hugefile | sed 's/old/new/g'

# Use grep to filter first (if applicable)
grep 'relevant' hugefile | sed 's/old/new/g'

Parallel Processing

# Split file, process in parallel, merge
split -n l/4 hugefile chunk_
parallel sed -i 's/old/new/g' ::: chunk_*
cat chunk_* > processed_file
rm chunk_*

# Or use GNU parallel with sed
cat hugefile | parallel --pipe sed 's/old/new/g' > output

Quick Reference Card

╔══════════════════════════════════════════════════════════════════════════════╗
║                         SED QUICK REFERENCE                                  ║
╠══════════════════════════════════════════════════════════════════════════════╣
║                                                                              ║
║  PREVIEW/APPLY PATTERN (Golden Rule)                                        ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  sed -n 's/old/new/gp' file        Preview changes                          ║
║  sed -i 's/old/new/g' file         Apply changes                            ║
║  sed -i.bak 's/old/new/g' file     Apply with backup                        ║
║                                                                              ║
║  COMMANDS                                                                    ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  s/old/new/g     Substitute (g=global)                                       ║
║  d               Delete line                                                 ║
║  p               Print line                                                  ║
║  i\text          Insert before                                               ║
║  a\text          Append after                                                ║
║  c\text          Change (replace) line                                       ║
║  q               Quit                                                        ║
║  y/abc/xyz/      Transform characters                                        ║
║                                                                              ║
║  ADDRESSES                                                                   ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  5               Line 5                                                      ║
║  5,10            Lines 5-10                                                  ║
║  5,$             Line 5 to end                                               ║
║  /pattern/       Lines matching pattern                                      ║
║  /start/,/end/   Range between patterns                                      ║
║  1~2             Every 2nd line from 1                                       ║
║                                                                              ║
║  FLAGS                                                                       ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  g   Global (all occurrences)                                                ║
║  p   Print                                                                   ║
║  i   Case-insensitive                                                        ║
║  2   Only 2nd occurrence                                                     ║
║                                                                              ║
║  ESCAPING (BRE)                                                              ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  \.  Literal dot           \*  Literal asterisk                              ║
║  \[  Literal bracket       \\  Literal backslash                             ║
║  \$  Literal dollar        \/  Literal slash (or use |#@ delimiter)          ║
║                                                                              ║
║  HOLD BUFFER                                                                 ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  h   Copy pattern→hold     H   Append pattern→hold                           ║
║  g   Copy hold→pattern     G   Append hold→pattern                           ║
║  x   Exchange pattern↔hold                                                   ║
║                                                                              ║
║  REGEX                                                                       ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  .         Any character        ^     Start of line                          ║
║  *         Zero or more         $     End of line                            ║
║  [abc]     Character class      [^ab] Negated class                          ║
║  \(\)      Capture group        \1    Backreference                          ║
║  &         Entire match                                                      ║
║                                                                              ║
║  COMMON OPERATIONS                                                           ║
║  ────────────────────────────────────────────────────────────────────────    ║
║  sed '/^$/d'                   Delete empty lines                            ║
║  sed '/^#/d'                   Delete comment lines                          ║
║  sed 's/^/#/'                  Comment out line                              ║
║  sed 's/^#//'                  Uncomment line                                ║
║  sed -n '10,20p'               Print lines 10-20                             ║
║  sed 's/  */ /g'               Collapse whitespace                           ║
║  sed 'N;s/\n/ /'               Join every 2 lines                            ║
║  sed '1!G;h;$!d'               Reverse lines (tac)                           ║
║                                                                              ║
╚══════════════════════════════════════════════════════════════════════════════╝

Related Documentation

Linux Operations

Cross-Site References

  • sed Operations Reference - Quick reference for infrastructure tasks

  • Domain Join Guide - sed usage for DC migration