INC-2026-04-06-001: Resolution

Proposed Change: CR-2026-04-07-iot-vpn-ipsec

Justification

Users on the Domus-IoT VLAN require the ability to connect to external work VPN (Palo Alto GlobalProtect). The current IOT_WAN policy permits the tunnel handshake (TCP 443) but blocks IPsec data transport protocols. This creates a misleading failure state where the VPN appears connected but no traffic flows.

The proposed rules add the minimum required protocols for IPsec VPN passthrough. These are outbound-only rules (IoT → WAN) and do not permit any inbound access. The security posture of the IoT zone is preserved — devices can only initiate outbound VPN connections, not receive inbound ones.

Proposed VyOS Configuration

configure

# Rule 80: Allow IPsec ESP (IP protocol 50) — encrypted data tunnel
set firewall ipv4 name IOT_WAN rule 80 action accept
set firewall ipv4 name IOT_WAN rule 80 description 'Allow IPsec ESP for VPN'
set firewall ipv4 name IOT_WAN rule 80 protocol esp

# Rule 85: Allow IPsec NAT-T (UDP 4501) — NAT traversal for IPsec
set firewall ipv4 name IOT_WAN rule 85 action accept
set firewall ipv4 name IOT_WAN rule 85 description 'Allow IPsec NAT-T for VPN'
set firewall ipv4 name IOT_WAN rule 85 destination port 4501
set firewall ipv4 name IOT_WAN rule 85 protocol udp

# Rule 90: Allow IKE (UDP 500) — IPsec key exchange
set firewall ipv4 name IOT_WAN rule 90 action accept
set firewall ipv4 name IOT_WAN rule 90 description 'Allow IKE for VPN'
set firewall ipv4 name IOT_WAN rule 90 destination port 500
set firewall ipv4 name IOT_WAN rule 90 protocol udp

# Also enable default-log on IOT_WAN for future visibility
set firewall ipv4 name IOT_WAN default-log

# Review changes before committing
compare

Risk Assessment

Risk Mitigation

Opens ESP/IPsec outbound from IoT

Outbound only — IoT initiates the connection. No inbound rules changed. Return traffic handled by existing established/related rule (rule 10).

Could allow unauthorized VPN use from IoT devices

IoT devices (cameras, smart home) don’t have VPN clients. This enables human users (laptops) temporarily on IoT VLAN. Monitor with default-log.

Blast radius if misconfigured

Changes only affect IOT_WAN (IoT → WAN direction). No other zone policies modified. Rollback: delete rules 80, 85, 90.

Rollback Procedure

configure
delete firewall ipv4 name IOT_WAN rule 80
delete firewall ipv4 name IOT_WAN rule 85
delete firewall ipv4 name IOT_WAN rule 90
compare
commit
save

Proof-of-Concept Test Procedure

Before committing the change, capture evidence that IOT_WAN is dropping IPsec traffic.

Step 1: Enable Drop Logging on IOT_WAN

configure
set firewall ipv4 name IOT_WAN default-log
commit

Step 2: Monitor Firewall Log in Real Time

# Terminal 1 on VyOS — watch for IOT_WAN drops
tail -f /var/log/messages | grep "IOT_WAN"

Step 3: User Connects VPN

Have the user on Domus-IoT connect GlobalProtect VPN. The firewall log should show:

[ipv4-NAM-IOT_WAN-default-D] SRC=10.50.40.xxx DST=<vpn-gateway-ip> PROTO=ESP
[ipv4-NAM-IOT_WAN-default-D] SRC=10.50.40.xxx DST=<vpn-gateway-ip> PROTO=UDP DPT=4501

This is the proof — ESP or UDP 4501 packets from the IoT client being dropped by IOT_WAN.

Step 4: Apply Fix and Retest

# Apply the proposed rules (from "Proposed VyOS Configuration" above)
# Then have user reconnect VPN
# Verify:
# - VPN connects AND data flows
# - Work resources accessible
# - No new drops in IOT_WAN log

Step 5: Verify and Save

# Confirm no new drops
show log firewall | grep "IOT_WAN" | tail -10

# If working, save
commit
save

# If not working, rollback (see Rollback Procedure)

Verification Checklist

  • Enable default-log on IOT_WAN

  • Capture firewall drops during VPN connection attempt (proof)

  • Apply rules 80, 85, 90

  • User connects VPN — tunnel AND data transport functional

  • User can access work resources through VPN

  • External sites reachable while VPN connected

  • DNS resolution works inside VPN tunnel

  • ping -c 100 through VPN — no packet loss

  • MTU test: ping -M do -s 1400 <work-resource> passes

  • Firewall log shows no new IOT_WAN drops for VPN traffic

  • save configuration to persist across reboot

Change Record

Change Record: CR-2026-04-07-iot-wan-vpn-passthrough — Complete

Four rules applied to VyOS firewall ipv4 name IOT_WAN:

  1. Rule 75 (Applied): TCP 2443 — GlobalProtect gateway tunnel

  2. Rule 80 (Applied): IPsec ESP (protocol 50) — encrypted data transport

  3. Rule 85 (Applied): UDP 4501 — IPsec NAT-T data channel

  4. Rule 90 (Applied): UDP 500 — IKE key exchange

  5. default-log enabled — all drops now logged for visibility

All rules verified 2026-04-07 12:10 PST. VPN connectivity restored.

API Queries — domus-api

Query this incident and its linked change record via the documentation REST API.

Start the API

cd ~/atelier/_projects/personal/domus-api && uv run uvicorn domus_api.main:app --host 0.0.0.0 --port 8080

Incident Queries

# Incident title
curl -s localhost:8080/pages/case-studies/incidents/INC-2026-04-06-domus-iot-vpn-connectivity | jq -r '.title'

# Incident content (readable plain text)
curl -s localhost:8080/pages/case-studies/incidents/INC-2026-04-06-domus-iot-vpn-connectivity | jq -r '.content' | head -80

# Search for all IOT_WAN references across the documentation system
curl -s 'localhost:8080/search?q=IOT_WAN' | jq -r '.results[] | "\(.title)\t\(.path)"'

# Search for VPN-related content
curl -s 'localhost:8080/search?q=GlobalProtect' | jq -r '.results[] | "\(.title)\t\(.path)"'

Change Record Queries

# CR title
curl -s localhost:8080/pages/case-studies/changes/CR-2026-04-07-iot-wan-vpn-passthrough | jq -r '.title'

# CR content
curl -s localhost:8080/pages/case-studies/changes/CR-2026-04-07-iot-wan-vpn-passthrough | jq -r '.content' | head -80

# List all change records
curl -s 'localhost:8080/pages?category=case-studies' | jq -r '.pages[] | select(.path | contains("changes")) | "\(.title)\t\(.path)"'

Traceability — Incident to CR

# Find all documents referencing this incident
curl -s 'localhost:8080/search?q=INC-2026-04-06-001' | jq -r '.results[] | {title, path}'

# Find the CR linked to this incident
curl -s 'localhost:8080/search?q=CR-2026-04-07-iot-wan' | jq -r '.results[] | {title, path}'

# Verify bidirectional link — incident references CR and CR references incident
curl -s localhost:8080/pages/case-studies/incidents/INC-2026-04-06-domus-iot-vpn-connectivity | jq -r '.content' | grep -i "CR-2026-04-07"
curl -s localhost:8080/pages/case-studies/changes/CR-2026-04-07-iot-wan-vpn-passthrough | jq -r '.content' | grep -i "INC-2026-04-06"

Export for Reporting

# Terminal table — all incidents
curl -s 'localhost:8080/pages?category=case-studies' | jq -r '.pages[] | select(.path | contains("incidents")) | [.title, .path] | @tsv' | column -t -s $'\t'

# Export incident as text file for email/ticket
curl -s localhost:8080/pages/case-studies/incidents/INC-2026-04-06-domus-iot-vpn-connectivity | jq -r '.content' > /tmp/INC-2026-04-06-001-report.txt

# Export CR as text file
curl -s localhost:8080/pages/case-studies/changes/CR-2026-04-07-iot-wan-vpn-passthrough | jq -r '.content' > /tmp/CR-2026-04-07-report.txt