Recovery Architecture

1. Overview

Multi-layer recovery architecture for Domus Digitalis infrastructure implementing a defense-in-depth backup strategy that survives hardware failure, ransomware, and physical disasters.

2. 3-2-1 Backup Strategy

Rule Meaning Implementation Recovery Time

3 copies

At least 3 copies of data

Hot (SSD) + Warm (NAS) + Cold (USB)

N/A

2 storage types

Different media types

SSD + HDD/NAS + Optical

N/A

1 offsite

Geographic separation

LUKS USB rotated offsite

Hours to days

+Archival

1000+ year durability

M-Disc in fireproof safe

Days

3. Backup Tiers

3.1. Tier 1: HOT (Primary)

Location Contents

Workstation SSD

~/.secrets/, ~/.ssh/, working repos

Git repositories

Encrypted .age files

Recovery: Instant - this is your working copy.

3.2. Tier 2: WARM (Automated)

Data NAS Path Backup Command

ISE configs

/ise_backups

netapi ise backup --upload-nas

WLC configs

/wlc_backups

netapi wlc backup --upload-nas

pfSense configs

/firewall_backups

netapi pfsense backup --upload-nas

IOS switch configs

/switch_backups

netapi ios backup --all --upload-nas

KVM VM definitions

/kvm_backups

netapi kvm backup --all --upload-nas

Keycloak realms

/Backups/keycloak

netapi keycloak backup --upload-nas

Workstation (Borg)

/Backups/borg

borg create

Schedule: Daily infrastructure, weekly workstation.

3.3. Tier 3: COLD (Offline)

Location Contents

LUKS USB #1 (home safe)

age key, SSH keys, GPG keys, LUKS headers

LUKS USB #2 (offsite)

Same as #1, rotated quarterly

Schedule: Monthly sync, quarterly rotation.

3.4. Tier 4: ARCHIVAL (M-Disc)

Location Contents

M-Disc (fireproof safe)

age master key, recovery passphrases (printed)

Schedule: Annual burn, verify readability.

4. Credential Chain

Recovery credentials must be restored in order:

Priority Credential Unlocks

1

age master.key

ALL .age encrypted secrets

2

SSH private keys

Access to all systems

3

dsec secrets

Service credentials (ISE, WLC, NAS, etc.)

4

YubiKey

Hardware-bound authentication

Without the age key, ALL encrypted data is permanently unrecoverable. Always maintain multiple backups.

5. Recovery Scenarios

Scenario Recovery Source Time

Lost workstation

LUKS USB → age key → SSH → dsec

1-2 hours

Lost YubiKey

Backup YubiKey or software fallback key

30 min

Corrupted LUKS header

Header backup on LUKS USB

15 min

Lost age key

M-Disc archival backup

1 hour

Infrastructure failure

NAS backups via netapi

1-4 hours

6. Quick Commands

6.1. Check Backup Status

dsource d000 dev/network
netapi synology backup-status --detailed

6.2. Run All Backups

netapi ise backup --upload-nas
netapi wlc backup --upload-nas
netapi pfsense backup --upload-nas
netapi ios backup --all --upload-nas
netapi kvm backup --all --upload-nas

6.3. Mount LUKS Backup USB

sudo cryptsetup luksOpen /dev/sdX1 backup-usb
sudo mount /dev/mapper/backup-usb /mnt/backup

6.4. Recover age Key (Critical First Step)

cp /mnt/backup/keys/master.age.key ~/.secrets/.metadata/keys/
chmod 600 ~/.secrets/.metadata/keys/master.age.key

7. Documentation Map

7.1. Strategy & Procedures

7.2. Recovery Components

7.3. Storage & Media

7.4. Verification & Drills