Backup and Recovery

Overview

Secrets backup follows a 5-tier defense-in-depth strategy that survives hardware failure, ransomware, and physical disasters.

Backup Architecture
Figure 1. Backup Architecture

The age master key is THE critical credential. Without it, ALL encrypted secrets are permanently unrecoverable. Maintain 3+ copies in different locations.

Backup Tiers

Tier Storage Contents Recovery Time

Tier 0: SNAPSHOT

Snapper (local btrfs)

Instant rollback of / and /home

Seconds

Tier 1: HOT

Workstation SSD

~/.secrets/, ~/.ssh/, Git repos

Instant

Tier 2: WARM

Borg → Synology NAS

Deduplicated, versioned backups

Minutes

Tier 3: COLD

Seagate Primary + Backup SSDs

LUKS + btrfs encrypted mirrors

Minutes

Tier 4: ARCHIVAL

Verbatim M-DISC (fireproof safe)

age key, LUKS headers, GPG keys

Days

Critical Recovery Items

LUKS Headers

LUKS headers are stored on the encrypted disk. Header corruption = permanent data loss. Always backup before any disk operation.

 
# Backup LUKS header
cryptsetup luksHeaderBackup /dev/nvme0n1p2 \
    --header-backup-file luks-header-$(hostname)-$(date +%Y%m%d).img

# Store on LUKS USB and M-Disc
cp luks-header-*.img /mnt/backup/headers/

age Master Key

# Key location
~/.secrets/.metadata/keys/master.age.key

# Backup locations (all 3+ copies REQUIRED)
# 1. LUKS USB #1 (home safe)
# 2. LUKS USB #2 (offsite)
# 3. M-Disc (fireproof safe)

SSH Keys

# YubiKey resident keys - regenerate from hardware
ssh-keygen -K

# Software fallback key - backup to LUKS USB
~/.ssh/id_ed25519_fallback

Recovery Procedures

Quick Recovery (Tier 1)

# Already on workstation - just use dsec
dsource d000 dev/network

Lost Workstation (Tier 3)

# 1. Mount LUKS backup USB
sudo cryptsetup luksOpen /dev/sdX1 backup-usb
sudo mount /dev/mapper/backup-usb /mnt/backup

# 2. Restore age master key
mkdir -p ~/.secrets/.metadata/keys
cp /mnt/backup/keys/master.age.key ~/.secrets/.metadata/keys/
chmod 600 ~/.secrets/.metadata/keys/master.age.key

# 3. Restore SSH keys (or regenerate from YubiKey)
ssh-keygen -K  # From YubiKey
# OR
cp /mnt/backup/keys/id_ed25519_fallback ~/.ssh/
chmod 600 ~/.ssh/id_ed25519_fallback

# 4. Clone secrets repo
git clone git@github.com:EvanusModestus/domus-secrets.git ~/.secrets/vaults

# 5. Test recovery
dsource d000 dev/network
netapi ise mnt sessions

Corrupted LUKS Header

# 1. Boot from Arch Linux ISO
# 2. Mount LUKS USB or M-Disc with header backup
sudo mount /dev/sdX1 /mnt/backup  # or mount optical drive

# 3. Restore header
cryptsetup luksHeaderRestore /dev/nvme0n1p2 \
    --header-backup-file /mnt/backup/headers/luks-header-hostname.img

# 4. Unlock with passphrase and continue boot

Complete Disaster (Tier 4)

# 1. Retrieve M-Disc from fireproof safe
# 2. Boot new hardware from Arch ISO
# 3. Mount optical drive
# 4. Restore age key and LUKS header
# 5. Rebuild from Git repositories

Backup Schedule

Frequency Action Verification

Weekly

Borg to NAS + Seagate SSD sync

Check borg info output

Monthly

Sync to LUKS USB #1

Test decrypt of one secret

Quarterly

Rotate LUKS USB #2 offsite

Verify both USBs readable

Annually

Burn new M-Disc + recovery drill

Full restore to VM

Comprehensive Documentation

For detailed procedures, see Infrastructure Operations:

Guide Description

Recovery Architecture (infra-ops)

Full system recovery architecture with dependency chain

Credential Chain (infra-ops)

Step-by-step unlock sequence from cold boot

LUKS Header Backup (infra-ops)

Complete LUKS header backup and restore procedures

M-Disc Archival (infra-ops)

Verbatim M-Disc burning and verification

Seagate Primary SSD (infra-ops)

Daily LUKS + btrfs backup with subvolumes

Seagate Backup SSD (infra-ops)

Weekly sync from primary for redundancy

Borg Backup (infra-ops)

Borg repository setup and maintenance

gocryptfs Vaults (infra-ops)

Vault recovery and re-encryption

Disaster Recovery Runbook (infra-ops)

Complete disaster recovery playbook

Backup Strategy (infra-ops)

3-2-1+Archival backup strategy overview

Related Pages