Daily Worklog

1. Overview

Date: 2026-02-03 (Tuesday)

Location: Remote

Focus: iPSK HA deployment, runbook creation, 290 GWN MAC import, Infoblox API, csvkit tooling, Antora build fixes, HOME runbook v2.0, CHLA runbook table refactor

Strategic Priorities:

  1. iPSK Manager HA deployment — runbook, diagrams, attribute alignment

  2. 290 GWN iPad MAC import into iPSK Manager

  3. Infoblox API credential verification

  4. csvkit tooling documentation

2. Session: iPSK Manager HA Documentation

2.1. Context

Full documentation push for iPSK Manager HA deployment at CHLA. Created operational runbook, updated architecture diagrams for both CHLA and HOME environments, aligned attribute naming across all iPSK projects with the established PRJ-ISE-CHLA-LINUX-ANTORA conventions.

2.2. Work Completed

2.2.1. 1. HOME iPSK HA Diagram Updated

File: PRJ-ISE-IPSK-HOME-ANTORA/docs/asciidoc/modules/ROOT/images/diagrams/ipsk-ha-architecture.d2

Updated from old single-ODBC pattern to match CHLA production HA architecture:

  • Dual ODBC sources (Primary + Failover)

  • Static iPSK fallback profile (tertiary safety net)

  • pfSense DNS failover (home equivalent of Netscaler)

  • Sponsor access flow through VIP

  • Backup storage (Synology NAS)

  • Async MySQL replication (Primary → Replica)

SVG and PNG regenerated successfully.

2.2.2. 2. CHLA iPSK HA Diagram Updated

File: PRJ-ISE-IPSK-CHLA-ANTORA/docs/asciidoc/modules/ROOT/images/diagrams/ipsk-ha-architecture.d2

Same HA pattern as HOME but with CHLA-specific components:

  • Netscaler VIP (instead of pfSense)

  • ISE Cluster with PSN, ODBC Source 1/2, Static iPSK Fallback

All three diagram locations regenerated (Antora HOME, Antora CHLA, Runbook CHLA).

2.2.3. 3. Antora Attribute Expansion (Both Sites)

Both antora.yml files expanded from ~10 attributes to 40+ each:

CHLA (PRJ-ISE-IPSK-CHLA-ANTORA/docs/asciidoc/antora.yml):

  • Network: domain, ise-domain

  • ISE role-based: ise-pan-ip, ise-pan-hostname, ise-mnt-ip, ise-mnt-hostname, ise-psn-primary/secondary

  • ISE server-specific: ise-ppan-ip/hostname, ise-span-ip/hostname (ODBC relevant)

  • iPSK VMs: hostnames, IPs, FQDNs, SSH alias, web URL, Netscaler VIP

  • Database: port, name, 4 user accounts with role comments

  • Wireless: SSIDs, VLANs

  • ISE policy: policy set, ODBC source, identity source, authz profile

  • File paths: MySQL, Apache, iPSK install, backup, NAS mount

HOME (PRJ-ISE-IPSK-HOME-ANTORA/docs/asciidoc/antora.yml):

  • Same structure adapted for home enterprise (10.50.1.x, 5 SSIDs, pfSense)

  • WLC policy profiles for each SSID

  • ISE replication subnet for MySQL GRANT statements

Critical Fix: Attribute naming aligned with PRJ-ISE-CHLA-LINUX-ANTORA conventions.

Previous error: used INTERNAL_DOMAIN placeholders and bare hostnames. Corrected to: domain: chla.usc.edu, ise-ppan-hostname: ppan.ise.chla.org (full FQDNs).

PPAN/SPAN are the correct ISE nodes for ODBC — MnT has no role in iPSK.

2.2.4. 4. CHLA iPSK Operational Runbook

File: PRJ-ISE-IPSK-CHLA-ANTORA/runbooks/ipsk-manager-operational-runbook.adoc

Comprehensive operational runbook structured to match the Linux workstation runbook:

  • Executive summary with deployment facts

  • Deployment status table with PASS/FAIL/PENDING styling

  • 5 deployment phases: VM provisioning, MySQL HA, security hardening, ISE policy, sponsor portal + MAC import

  • "Action Required (Person):" callouts throughout

  • Role-based deployment checklists for Ben Castillo (SysEng), Samuel John (Database Architect, Digital Dev & Solutions Architecture), Argam Darbinian (Endpoint Engineer I), Evan, Network Team

  • Failover procedures (3 scenarios)

  • Backup and restore procedures

  • Troubleshooting section (4 subsections)

  • Document revision history and classification footer

Build output: HTML (108K), PDF (409K)

2.2.5. 5. Attribute Naming Correction

User caught inconsistent attribute naming between iPSK and Linux projects.

Before (wrong):

@INFOBLOX_GM_IP = INTERNAL_DOMAIN
ise-ppan-hostname: ppan  (bare hostname)

After (correct, matches PRJ-ISE-CHLA-LINUX-ANTORA):

domain: chla.usc.edu
ise-ppan-hostname: ppan.ise.chla.org  (full FQDN)

MnT references removed from ODBC sections — ODBC is configured on PPAN (10.101.2.121), replicated to SPAN (10.101.2.122).

3. Session: 290 GWN iPad MAC Import

3.1. Context

Argam Darbinian (Endpoint Engineer I) provided 290 GWN iPad MAC addresses exported from Airwatch/Intune for import into the iPSK Manager. These are IoT devices connecting to the CHLA_IoT SSID.

3.2. Work Completed

3.2.1. MAC Address Preparation

  • Received raw MAC list from Argam (Airwatch/Intune export)

  • Reformatted CSV: added colon-separated MAC format

  • Appended .inside.domusdigitalis.dev to serial numbers for FQDN identification

nvim regex for MAC formatting (applied to 290 rows):
:2,$s/\(\x\x\)\(\x\x\)\(\x\x\)\(\x\x\)\(\x\x\)\(\x\x\)$/\1:\2:\3:\4:\5:\6/
:2,$s/^[^,]\+/\0.chla.usc.edu/

3.2.2. Import Result

Metric Value

Devices Imported

290

Device Type

GWN iPads

Source

Argam Darbinian (Endpoint Engineer I) (Airwatch/Intune export)

Date

2026-02-03

Target SSID

CHLA_IoT

iPSK Manager

ipsk-mgr-01.inside.domusdigitalis.dev

3.2.3. Runbook Updated

  • Deployment status table: MAC import marked DONE

  • Argam’s checklist: export, provide, verify all checked off

  • Evan’s checklist: prepare CSV and import checked off

  • Revision history: v1.1.1 entry documenting the import

4. Session: Infoblox API Configuration Check

4.1. Context

Team requesting Infoblox data pulls. Checked netapi Infoblox configuration to verify admin credential API access rights.

4.2. Findings

netapi Infoblox client code is well-structured:

  • client.py — InfobloxClient with WAPI REST integration

  • infoblox.py — CLI with commands: get-networks, get-host-records, get-a-records, get-leases, get-reservations, search-ip, get-next-ip, create-host, create-a-record, create-reservation

  • Environment variables: INFOBLOX_HOST, INFOBLOX_USER, INFOBLOX_PASS

Problem: Credentials are all placeholders in both d001/dev and d001/prod secrets:

@INFOBLOX_GM_IP = <INFOBLOX_IP>
@INFOBLOX_API_USER = <INFOBLOX_USER>
@INFOBLOX_API_PASS = <INFOBLOX_PASSWORD>

4.3. Action Required

  • Get Infoblox Grid Master IP from network team

  • Get admin username and password with API access

  • Update encrypted secrets: ~/.secrets/environments/domains/d001/dev/network.env.age

  • Test: dsource d001 dev/network && netapi infoblox get-networks

5. Session: csvkit Reference Documentation

5.1. Context

Needed csvkit (csvlook, csvcut, csvgrep, etc.) for MAC address CSV preparation. Created reference document for ongoing use.

5.2. Deliverable

File: 02_Assets/ARS-DEVTOOLS-PYTHON/2026-PY-011-csvkit-Reference.adoc

Comprehensive reference covering all csvkit tools:

  • csvlook — render CSV as table

  • csvcut — select/reorder columns

  • csvgrep — filter rows (regex, inverse)

  • csvsort — sort rows

  • csvstat — summary statistics

  • csvjson — CSV to JSON

  • in2csv — Excel/JSON to CSV

  • csvjoin — join two CSVs

  • csvstack — concatenate CSVs

  • csvsql — SQL queries on CSV

  • sql2csv — database to CSV

  • csvformat — reformat delimiter/quoting

  • csvclean — validate and fix

  • Common pipelines section

Install: uv tool install csvkit (global, no venv needed)

6. Session: Antora Site Build Fixes

6.1. Context

Both HOME and CHLA Antora sites had build errors/warnings accumulated during rapid documentation expansion. Systematic fix pass to get both sites to 0 errors, 0 warnings.

6.2. Work Completed

6.2.1. 1. CHLA Antora — List Numbering Fix (Stream 2)

File: PRJ-ISE-CHLA-LINUX-ANTORA/docs/asciidoc/modules/ROOT/pages/05-appendix/troubleshooting.adoc

Fixed 7 build warnings caused by numbered list with bold section headers ("Before/During/After PKI Migration") breaking list continuity. Converted to 3 separate numbered sub-lists under bold headers.

6.2.2. 2. HOME Antora — 26 xref Fixes (Stream 1)

Systematic fix of all broken cross-references across 8 files:

  • 15 missing subdirectory prefix — Antora xrefs are relative to pages/ root, not filesystem-relative. Added 04-linux-client/, 03-ise-config/, 99-appendix/ prefixes.

  • 8 wrong ../ prefix — Removed filesystem-relative ../ from xrefs to other subdirectories.

  • 2 dead cross-project links — Removed references to non-existent projects (cli/ise/ers/authz-rules.adoc, PRJ-INFRA-OPS-ANTORA).

  • 1 include escape — Backslash-escaped include::example$script.sh[] directives inside listing blocks to prevent Asciidoctor preprocessor resolution.

Files modified:

  • hardened-dacl.adoc (3 xrefs)

  • domain-join.adoc (1 xref)

  • networkmanager-wifi.adoc (6 xrefs)

  • networkmanager-wired.adoc (4 xrefs)

  • privilege-separation.adoc (3 xrefs)

  • workstation-status.adoc (5 xrefs)

  • troubleshooting.adoc (1 dead link)

  • byod-operations-runbook.adoc (1 dead link)

  • validation-scripts.adoc (include escapes)

6.2.3. 3. HOME Antora — 6 d2 Diagrams Created (Stream 1)

Created architecture diagrams in modules/ROOT/images/diagrams/:

Diagram Content

privilege-separation.d2

Zero-trust privilege model: AD → SSSD → PAM → privilege tiers

posture-compliance-flow.d2

ISE posture lifecycle: connect → discovery → agent check → compliant/quarantine

pki-trust-chain.d2

ROOT CA → ISE trust + Linux trust → mutual authentication

deployment-architecture.d2

Home enterprise topology: pfSense → switch → ISE/DC/NAS → workstations

certificate-chain.d2

Certificate trust chain: ROOT CA → server/client certs → validation

dacl-processing.d2

ACL processing: deny RFC1918 → permit essential → permit internet → deny log

Each compiled to both SVG and PNG. All referenced from existing Antora pages.

6.2.4. 4. Build Verification

Both Antora sites verified clean: 0 errors, 0 warnings each.

7. Session: HOME Runbook v2.0 Enhancement (Stream 3)

7.1. Context

HOME enterprise runbook (linux-eaptls-deployment-runbook.adoc) was at v1.1 (642 lines). Enhanced to match CHLA runbook quality, adapted for self-managed Arch Linux environment.

7.2. Work Completed

  • Added ISE POLICY OBJECTS attribute section (policy set, authz profiles, dACLs, endpoint groups, AD groups)

  • Added WORKSTATIONS attribute section (per-device hostname, model, MAC, interface)

  • Added deployment status table with PASS/FAIL/PENDING CSS styling

  • Added Phase 1.5: ISE Pre-Deployment Validation (endpoint groups, registration, authz profiles, dACLs, rule ordering via netapi)

  • Added Phase 4: ISE Session Verification and Policy Transition (MnT session check, switch-side auth, CoA, dACL enforcement)

  • Added certificate-key match verification step (modulus comparison)

  • Expanded troubleshooting: ISE session diagnostics, wrong VLAN, dACL not applying, cert chain validation

  • Added ISE error code reference table (12514, 12321, 22056, 22045, 22059)

  • Added netapi quick reference command table (8 commands)

  • v1.1 (642 lines) → v2.0 (1026 lines), 127K HTML

8. Session: CHLA Runbook Table Refactor

8.1. Context

CHLA deployment status tables had combined "Owner / Notes" columns. Split into separate columns for clearer accountability tracking across all three runbook files.

8.2. Work Completed

8.2.1. 1. Weekly Status (weekly-status-2026-02-03.adoc)

Changed from [cols="3,1,3"] to [cols="3,1,2,3"]. Split combined "Owner / Notes" header into separate "Owner" and "Notes" columns. All 17 rows updated with team/person attribute references.

8.2.2. 2. Deployment Runbook (linux-workstation-deployment-runbook.adoc)

Changed Validation Summary from [cols="3,1,4"] to [cols="3,1,2,3"]. Added Owner column with team/person attribution. Added ISE MAB and EAP-TLS policy rows (PASS, Information Security Team). Reordered: PASS items grouped first, then FAIL, then PENDING.

8.2.3. 3. Meeting Prep (meeting-prep-linux-deployment.adoc)

Added personnel/team attributes section, software version attributes, CSS styling block, and full 4-column "Current Deployment State" table matching the other two files.

8.3. Communication Log Update

Added Sarah Clizer (CISO)'s Friday 3pm hard deadline directive and Dr. Shahab Asgharzadeh’s workstation readiness inquiry with response to the weekly status document.

9. Session: Tooling Advisory — pass vs gopass

Evaluated whether pass (Unix password manager) can be safely removed now that gopass is the active password store.

Finding: gopass 1.16.1 is the active store, using ~/.password-store natively. pass 1.7.4 is installed but unused — only pass-otp depends on it, and gopass otp provides the same functionality. No shell configs or other packages reference pass. Safe to remove with pacman -R pass-otp pass.

10. Day Summary

Task Status Notes

HOME iPSK HA diagram

DONE

Updated to match CHLA production pattern

CHLA iPSK HA diagram

DONE

Netscaler VIP, dual ODBC, static fallback

Both antora.yml attribute expansion

DONE

40+ attributes each, comprehensive comments

Attribute naming alignment

DONE

Matched PRJ-ISE-CHLA-LINUX-ANTORA conventions exactly

CHLA iPSK operational runbook

DONE

108K HTML, 409K PDF, phased deployment structure

290 GWN MAC import

DONE

Argam Darbinian (Endpoint Engineer I) provided, imported into iPSK Manager

Infoblox netapi check

BLOCKED

Credentials are placeholders — need GM IP and admin creds

csvkit reference doc

DONE

ARS-DEVTOOLS-PYTHON/2026-PY-011

Monthly log updated

DONE

iPSK project links, day-03 include enabled

CHLA Antora list numbering fix

DONE

7 warnings resolved in troubleshooting.adoc

HOME Antora xref fixes

DONE

26 broken xrefs fixed across 9 files

HOME Antora d2 diagrams

DONE

6 diagrams created (SVG + PNG)

HOME runbook v2.0

DONE

642 → 1026 lines, ISE policy integration

CHLA runbook table refactor

DONE

3 files: 4-column status tables (Owner separated)

CHLA weekly status updates

DONE

Sarah deadline, Shahab response, communication log

pass/gopass evaluation

DONE

Safe to remove pass — gopass covers all functionality

10.1. Commits

Hash Message

dd39e316

[docs] Add csvkit command reference (ARS-DEVTOOLS-PYTHON)

17909a4d

[docs] Restructure iPSK runbook with phased deployment, status tracking, role assignments

605b87e9

[docs] iPSK: mark 290 GWN device MAC import complete (2026-02-03)

4a194833

[docs] CHLA Antora: fix list numbering warnings in troubleshooting.adoc

34b39058

[docs] HOME Antora: fix all build errors, add 6 d2 diagrams

9eba5f24

[docs] HOME runbook v2.0: ISE policy attributes, deployment status, expanded troubleshooting

9ef1283d

[docs] Weekly status: add Friday 3pm hard deadline per Sarah, update Shahab response

0653d208

[docs] Weekly status: clean up Shahab response as copiable code block

88f17ba5

[docs] CHLA runbooks: split status tables into 4-column format

10.2. Pending (Carry Forward)

  • Infoblox: get GM IP and admin credentials, update d001/dev secrets

  • iPSK pages: attribute substitution pass (replace remaining hardcoded values)

  • SSIDs may have changed (user mentioned, not yet specified new names)

  • Sponsor portal self-service configuration

  • Remove pass + pass-otp packages (pacman -R pass-otp pass)

11. Tags

ipsk ipsk-ha mysql-replication odbc antora runbook diagrams d2 csvkit infoblox gwn-ipads mac-import attribute-alignment ise-policy xref home-enterprise chla gopass pass eap-tls dacl posture

12. Document Revision History

Version Date Changes

1.0

2026-02-03

Initial daily worklog — iPSK HA documentation, 290 GWN MAC import, Infoblox check, csvkit reference

1.1

2026-02-03

Added afternoon sessions: Antora build fixes (both sites), HOME runbook v2.0, CHLA runbook table refactor, pass/gopass evaluation