INC-2026-04-07-001: SSH Access Failure to modestus-razer

Incident Summary

Field	Value
Detected	2026-04-07 06:30 PST (manual discovery during Hyprland troubleshooting)
Mitigated	2026-04-07 06:45 PST (reverse SSH tunnel established)
Resolved	2026-04-07 06:50 PST (authenticated with correct identity file)
Duration	~20 minutes
Severity	P3 (Medium) — Workaround available
Impact	Cannot manage modestus-razer remotely from modestus-p16g. Razer stuck in TTY (Hyprland down), SSH required for remote troubleshooting.
Root Cause	Three compounding failures: (1) VyOS firewall blocks VLAN 50→VLAN 10 SSH by design, (2) SSH keys use custom suffixed paths not matched by default lookup, (3) SSH agent empty after reboot.

Field

Value

Detected

2026-04-07 06:30 PST (manual discovery during Hyprland troubleshooting)

Mitigated

2026-04-07 06:45 PST (reverse SSH tunnel established)

Resolved

2026-04-07 06:50 PST (authenticated with correct identity file)

Duration

~20 minutes

Severity

P3 (Medium) — Workaround available

Impact

Cannot manage modestus-razer remotely from modestus-p16g. Razer stuck in TTY (Hyprland down), SSH required for remote troubleshooting.

Root Cause

Three compounding failures: (1) VyOS firewall blocks VLAN 50→VLAN 10 SSH by design, (2) SSH keys use custom suffixed paths not matched by default lookup, (3) SSH agent empty after reboot.

Timeline

Time (PST) Event

Time (PST)	Event
06:23	Hyprland failure detected on modestus-razer. User drops to TTY.
06:25	Razer can SSH to P16g (VLAN 10→50 allowed). P16g cannot SSH to Razer (VLAN 50→10 denied by VyOS).
06:30	`ping 10.50.1.106` — 100% packet loss. Confirms firewall blocking ICMP and SSH cross-VLAN.
06:35	Reverse SSH tunnel established from Razer TTY: `ssh -R 2222:localhost:22 evanusmodestus@modestus-p16g`
06:38	`ssh -p 2222 localhost` — host key conflict. Stale `known_hosts:45` entry for `[localhost]:2222` from previous tunnel session.
06:39	`ssh-keygen -R '[localhost]:2222'` — removed stale key. Retried. Host key accepted.
06:40	`Permission denied (publickey,gssapi-with-mic)` — authentication failure.
06:41	`sudo ssh -p 2222 localhost` — attempted root auth. Wrong approach: root’s keypair, not user’s.
06:43	`ssh -vvv` debug: `identity file .ssh/id_ed25519 type -1` — no key files at default paths. Agent contains no identities.
06:45	`ls ~/.ssh/*.pub` — 16 keys found, all with custom suffixes (`_d000`, `_github`, `_sk_rk_d000`, etc.). None at default `id_ed25519` path.
06:48	`ssh -p 2222 -i ~/.ssh/id_ed25519_d000 localhost` — passphrase accepted, connection established.
06:50	Authenticated on modestus-razer. Incident resolved.