systemd Patterns

systemd patterns I’ve actually used. Every entry has a date and context.

2026-04-02: ACPI GPE Interrupt Storm Masking

Problem: System log flooded with ACPI GPE6E interrupts, causing CPU usage spikes on first boot after fresh Arch install.

Context: P16g ThinkPad P16g Gen 3 (Intel HX + RTX 5090). ACPI firmware fires GPE6E repeatedly — a known ThinkPad firmware quirk.

The Fix:

# Identify the storm
dmesg | grep -i gpe
# Or check syslog
grep -c "ACPI" /var/log/syslog

# Mask the GPE in the boot entry
# In /boot/efi/loader/entries/arch.conf, add to options line:
#   acpi_mask_gpe=0x6E

Rule: ThinkPad ACPI GPE6E storms are hardware-specific firmware quirks. Mask with kernel parameter acpi_mask_gpe=0x6E. Check dmesg | grep -i gpe to identify the specific GPE number — it varies by model.

Worklog: WRKLOG-2026-04-02

2026-04-02: mkinitcpio HOOKS Order Matters

Problem: System fails to boot after Arch install — LUKS passphrase prompt never appears. Default HOOKS line doesn’t include encrypt or btrfs.

Context: P16g deployment, dual-LUKS + btrfs setup. The encrypt hook MUST come before btrfs and filesystems in the HOOKS array.

The Fix:

# DRY RUN — preview the replacement
sed 's/^HOOKS=.*/HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt btrfs filesystems fsck)/' \
    /etc/mkinitcpio.conf | grep '^HOOKS'

# APPLY
sed -i 's/^HOOKS=.*/HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt btrfs filesystems fsck)/' \
    /etc/mkinitcpio.conf

# VERIFY
grep '^HOOKS' /etc/mkinitcpio.conf

# Regenerate all initramfs images
mkinitcpio -P

Rule: HOOKS order: keyboard keymap before block (so you can type the passphrase), encrypt before btrfs and filesystems (so LUKS opens before mount). Always dry-run sed without -i first, then apply, then verify.

Worklog: WRKLOG-2026-04-02

2026-04-02: Enable Fallback Presets in mkinitcpio

Problem: Fallback boot entry shows "initramfs not found" — Arch ships with fallback presets commented out in /etc/mkinitcpio.d/linux.preset.

Context: P16g deployment, systemd-boot with 3 entries (main, fallback, LTS). Fallback is the safety net when autodetect misses a driver.

The Fix:

# Comment out default-only, uncomment the one with fallback
sed -i "s/^PRESETS=('default')/#PRESETS=('default')/" /etc/mkinitcpio.d/linux.preset
sed -i "s/^#PRESETS=('default' 'fallback')/PRESETS=('default' 'fallback')/" /etc/mkinitcpio.d/linux.preset
sed -i 's/^#fallback_image/fallback_image/' /etc/mkinitcpio.d/linux.preset
sed -i 's/^#fallback_options/fallback_options/' /etc/mkinitcpio.d/linux.preset

# Same for LTS
sed -i "s/^PRESETS=('default')/#PRESETS=('default')/" /etc/mkinitcpio.d/linux-lts.preset
sed -i "s/^#PRESETS=('default' 'fallback')/PRESETS=('default' 'fallback')/" /etc/mkinitcpio.d/linux-lts.preset
sed -i 's/^#fallback_image/fallback_image/' /etc/mkinitcpio.d/linux-lts.preset
sed -i 's/^#fallback_options/fallback_options/' /etc/mkinitcpio.d/linux-lts.preset

# Regenerate — now produces 4 images (default + fallback for both kernels)
mkinitcpio -P

# Verify all 4 exist
ls -lh /boot/initramfs-*

Rule: Arch comments out fallback presets by default. Enable them for both linux.preset and linux-lts.preset before generating initramfs. Without fallback, you have no recovery option when autodetect misses a module.

Worklog: WRKLOG-2026-04-02

2026-04-02: Pacman Hook for ESP Kernel Sync

Problem: After kernel or NVIDIA update, systemd-boot still loads the old kernel because it reads from the ESP (VFAT), but pacman installs to /boot (ext4).

Context: P16g deployment, systemd-boot with separate ESP and /boot partitions. Kernels must be copied to ESP after every update.

The Fix:

sudo mkdir -p /etc/pacman.d/hooks

sudo tee /etc/pacman.d/hooks/99-esp-kernel-sync.hook << 'EOF'
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Target = usr/lib/modules/*/vmlinuz
Target = usr/lib/initcpio/*
Target = boot/*

[Action]
Description = Syncing kernels and initramfs to ESP...
When = PostTransaction
Exec = /bin/sh -c 'cp /boot/vmlinuz-* /boot/efi/ && cp /boot/initramfs-linux.img /boot/initramfs-linux-lts.img /boot/intel-ucode.img /boot/efi/'
EOF

Rule: When ESP and /boot are separate partitions, a pacman hook is mandatory for kernel sync. Without it, every pacman -Syu that touches a kernel requires manual cp to the ESP. Only copy default initramfs (not fallback) to keep the 512M ESP under capacity.

Worklog: WRKLOG-2026-04-02

2026-04-02: systemd-boot Entry with LUKS UUID Variable

Problem: Boot entries need LUKS partition UUIDs — hardcoding is error-prone and non-portable.

Context: P16g deployment, 3 systemd-boot entries (main, fallback, LTS) all referencing the same LUKS root UUID.

The Fix:

# Capture UUID once — use in all entries
ROOT_LUKS_UUID=$(blkid -s UUID -o value /dev/nvme0n1p3)
echo "Root LUKS UUID: $ROOT_LUKS_UUID"

# Write boot entry using the variable (NOTE: unquoted EOF so variable expands)
cat > /boot/efi/loader/entries/arch.conf << EOF
title   Arch Linux
linux   /vmlinuz-linux
initrd  /intel-ucode.img
initrd  /initramfs-linux.img
options cryptdevice=UUID=$ROOT_LUKS_UUID:cryptroot root=/dev/mapper/cryptroot rootflags=subvol=@ rw nvidia_drm.modeset=1 mem_sleep_default=s2idle
EOF

# Verify UUID expanded (not literal "$ROOT_LUKS_UUID")
grep 'cryptdevice' /boot/efi/loader/entries/arch.conf

Rule: Capture UUIDs into variables with blkid -s UUID -o value. Use unquoted EOF in heredocs so variables expand. Always verify the written file contains the actual UUID, not the variable name.

Worklog: WRKLOG-2026-04-02