API CLI Mastery

Six levels from curl | jq '.' to concurrent httpx audits and cross-endpoint pipeline analysis. Every example runs against domus-api (localhost:8080).

Level 1: curl + jq Foundations

HTTP Verbs and Response Inspection

# GET with status code — the -w flag writes metadata after the response
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/

# Pretty print any JSON endpoint
curl -s localhost:8080/stats | jq '.'

# Extract a single scalar
curl -s localhost:8080/stats | jq '.total_pages'

# Extract nested fields
curl -s localhost:8080/ | jq '.counts.standards'

# Array length — how many standards exist
curl -s localhost:8080/standards | jq '.standards | length'

# Select specific fields from array objects
curl -s localhost:8080/standards | jq '.standards[] | {id, title, status}'

# Raw output for piping — -r strips the quotes
curl -s localhost:8080/standards | jq -r '.standards[].id'

Why -s and -r Matter

-s (silent): suppresses curl’s progress meter. Without it, stderr pollutes your pipes. -r (raw): strips JSON string quotes. "STD-001" becomes STD-001. Essential when feeding jq output into awk/sort/uniq.

Level 2: Filtering and Transformation

select(), map(), sort_by(), group_by()

# Filter: only Active standards
curl -s localhost:8080/standards | jq '[.standards[] | select(.status == "Active")]'

# Invert filter: standards NOT Active (needs review)
curl -s localhost:8080/standards | jq '[.standards[] | select(.status != "Active")]'

# Tab-separated output for terminal consumption
curl -s localhost:8080/standards | jq -r '.standards[] | "\(.id)\t\(.status)\t\(.title)"'

# Sort by field
curl -s localhost:8080/standards | jq '.standards | sort_by(.id)'

# Reverse sort
curl -s localhost:8080/standards | jq '.standards | sort_by(.id) | reverse'

# Count by status using group_by
curl -s localhost:8080/standards | \
  jq '[.standards[].status] | group_by(.) | map({status: .[0], count: length})'

# @tsv for clean tab-separated output (use with column -t)
curl -s localhost:8080/standards | \
  jq -r '.standards[] | [.id, .status, .title] | @tsv' | column -t

# @csv for spreadsheet export
curl -s localhost:8080/standards | \
  jq -r '.standards[] | [.id, .status, .domain, .title] | @csv'

Level 3: Pipeline Composition — curl | jq | awk | sort | uniq

The Unix philosophy: each tool does one thing. jq extracts, awk reshapes, sort orders, uniq counts.

Cross-Endpoint Analysis

# Category distribution — which categories have the most pages
curl -s localhost:8080/stats | \
  jq -r '.categories | to_entries[] | "\(.value)\t\(.key)"' | sort -rn

# Top 5 education subcategories
curl -s localhost:8080/education | \
  jq -r '.subcategories | to_entries[] | "\(.value)\t\(.key)"' | sort -rn | head -5

# Search results by category — extract paths, awk splits on /, count
curl -s 'localhost:8080/search?q=security&limit=50' | \
  jq -r '.results[].path' | awk -F/ '{print $1}' | sort | uniq -c | sort -rn

# Codex categories with fewer than 5 entries (knowledge gaps)
curl -s localhost:8080/codex | \
  jq -r '.categories | to_entries[] | select(.value < 5) | "\(.key): \(.value)"'

Process Substitution — diff Two API Responses

# Compare standard IDs from /standards vs /pages?category=standards
diff <(curl -s localhost:8080/standards | jq -r '.standards[].id' | sort) \
     <(curl -s 'localhost:8080/pages?category=standards' | \
       jq -r '.pages[].path' | awk -F/ '{print $NF}' | sort)

# Before/after comparison — save state, make a change, compare
curl -s localhost:8080/projects | jq '.projects[].path' | sort > /tmp/before.json
# ... create a project ...
curl -s localhost:8080/projects | jq '.projects[].path' | sort > /tmp/after.json
diff /tmp/before.json /tmp/after.json

The awk Advantage

# Why awk over cut: awk handles variable whitespace and complex extraction
curl -s localhost:8080/standards | \
  jq -r '.standards[] | "\(.id) \(.status) \(.title)"' | \
  awk '$2 != "Active" {printf "%-10s %-12s %s\n", $1, $2, substr($0, index($0,$3))}'

# Backlink analysis with awk aggregation
curl -s localhost:8080/graph/references/standards/operations/change-control | \
  jq -r '.referenced_by[] | "\(.category)\t\(.title)"' | \
  awk -F'\t' '{cats[$1]++; titles[$1] = titles[$1] ? titles[$1] ", " $2 : $2} END {for (c in cats) printf "%s (%d): %s\n", c, cats[c], titles[c]}'

Level 4: Advanced jq — reduce, env, functions, recursive descent

reduce for Aggregation

# Total files across all example categories
curl -s localhost:8080/examples | \
  jq '.categories | to_entries | reduce .[] as $item (0; . + $item.value)'

# Running total with intermediate output
curl -s localhost:8080/stats | \
  jq '.categories | to_entries | reduce .[] as $item (
    {total: 0, breakdown: []};
    .total += $item.value |
    .breakdown += [{category: $item.key, count: $item.value, running: .total}]
  )'

Custom Functions with def

# Define a formatter and reuse it
curl -s localhost:8080/codex | \
  jq 'def fmt: "\(.value) entries"; .categories | to_entries[] | "\(.key): \(.value | fmt)"'

# Conditional formatting with if-then-else
curl -s localhost:8080/standards | \
  jq -r '.standards[] | "\(.id) [\(if .status == "Active" then "OK" else "REVIEW" end)] \(.title)"'

Environment Variables with --arg and $ENV

# Parameterize queries — pass shell variables into jq
QUERY="mandiant"
curl -s "localhost:8080/search?q=${QUERY}" | \
  jq --arg q "$QUERY" '{query: $q, hits: .total, top_3: [.results[:3][] | .title]}'

# Pass numeric arguments
curl -s localhost:8080/codex | \
  jq --argjson min 5 '.categories | to_entries[] | select(.value >= $min)'

# Access environment directly (jq 1.7+)
export API="localhost:8080"
jq -n 'env.API'

Recursive Descent (..)

# Find every "title" field anywhere in a nested response
curl -s localhost:8080/case-studies | jq '[.. | .title? // empty] | unique'

# Find all string values matching a pattern
curl -s localhost:8080/ | jq '[.. | strings | select(test("domus"))]'

# Get all paths to numeric values
curl -s localhost:8080/stats | jq '[path(.. | numbers)] | map(join("."))'

Level 5: httpx — Python Async HTTP Client

Basic Patterns

import asyncio
import httpx

BASE = "http://localhost:8080"

async def main():
    async with httpx.AsyncClient(base_url=BASE) as client:
        # Single request
        r = await client.get("/stats")
        print(f"Total pages: {r.json()['total_pages']}")

        # Concurrent requests — asyncio.gather fires all at once
        endpoints = ["/standards", "/projects", "/patterns", "/codex"]
        tasks = [client.get(ep) for ep in endpoints]
        responses = await asyncio.gather(*tasks)
        for ep, resp in zip(endpoints, responses):
            data = resp.json()
            count = data.get("count", data.get("total", data.get("total_entries", "?")))
            print(f"{ep}: {count}")

        # POST with JSON body
        incident = await client.post("/case-studies/incidents", json={
            "severity": "P4",
            "description": "httpx test incident",
        })
        print(f"Created: {incident.json()['id']}")

asyncio.run(main())

Concurrent Audit — Hit Every Endpoint

import asyncio
import httpx

ENDPOINTS = [
    "/", "/stats", "/attributes", "/standards",
    "/pages", "/search?q=security", "/projects",
    "/case-studies", "/skills", "/worklogs",
    "/patterns", "/codex", "/sessions",
    "/education", "/portfolio", "/reference",
    "/templates", "/api-hub", "/objectives",
    "/discoveries", "/runbooks", "/meta",
    "/drafts", "/trackers", "/partials", "/examples",
]

async def audit():
    async with httpx.AsyncClient(base_url="http://localhost:8080", timeout=10.0) as client:
        tasks = [client.get(ep) for ep in ENDPOINTS]
        results = await asyncio.gather(*tasks, return_exceptions=True)

        for ep, result in zip(ENDPOINTS, results):
            if isinstance(result, Exception):
                print(f"FAIL {ep}: {result}")
            else:
                status = "OK" if result.status_code == 200 else f"HTTP {result.status_code}"
                print(f"{status:>8}  {ep}")

asyncio.run(audit())

Level 6: Real-World Scenarios

Documentation Audit

# Find standards needing review
curl -s localhost:8080/standards | \
  jq -r '.standards[] | select(.status != "Active") | "\(.id): \(.status) — \(.title)"'

# Knowledge gap analysis — codex categories with sparse coverage
curl -s localhost:8080/codex | \
  jq -r '.categories | to_entries[] | select(.value < 5) | "\(.key): \(.value) entries"'

# Orphan detection — pages with no backlinks
# (requires checking each page against /graph/references)

Create-Then-Query Chain

# Create incident, then find related documentation
curl -s -X POST localhost:8080/case-studies/incidents \
  -H 'Content-Type: application/json' \
  -d '{"severity":"P3","description":"DNS resolution timeout on internal resolvers"}' | \
  jq -r '.id' | xargs -I{} sh -c \
    'echo "Created: {}"; curl -s "localhost:8080/search?q=dns&limit=5" | jq -r ".results[] | .title"'

Generate a Status Report from API Data

{
  echo "# domus-api Status Report — $(date +%Y-%m-%d)"
  echo ""

  echo "## Counts"
  curl -s localhost:8080/stats | \
    jq -r '"- Pages: \(.total_pages)\n- Partials: \(.total_partials)\n- Standards: \(.standards_count)\n- Projects: \(.projects_count)"'
  echo ""

  echo "## Standards Needing Review"
  curl -s localhost:8080/standards | \
    jq -r '.standards[] | select(.status != "Active") | "- \(.id): \(.title) (\(.status))"'
  echo ""

  echo "## Top 5 Content Categories"
  curl -s localhost:8080/stats | \
    jq -r '.categories | to_entries | sort_by(-.value)[:5][] | "- \(.key): \(.value) pages"'
} > /tmp/status-report.md

API Diff Over Time

# Snapshot current state
curl -s localhost:8080/stats | jq -S '.' > /tmp/stats-$(date +%Y%m%d).json

# Compare with previous snapshot
diff <(jq -S '.' /tmp/stats-20260406.json) <(jq -S '.' /tmp/stats-20260407.json)

# Structural diff — what keys changed
diff <(jq 'paths | join(".")' /tmp/stats-20260406.json | sort) \
     <(jq 'paths | join(".")' /tmp/stats-20260407.json | sort)

Batch Operations with xargs

# Get all pattern domains, then fetch each one
curl -s localhost:8080/patterns | \
  jq -r '.taxonomy | to_entries[].value.domains | keys[]' | sort -u | \
  xargs -I{} sh -c 'echo "=== {}"; curl -s "localhost:8080/patterns/{}" | jq ".count"'

# Parallel: fetch all codex categories concurrently (-P4 = 4 at a time)
curl -s localhost:8080/codex | \
  jq -r '.categories | keys[]' | \
  xargs -P4 -I{} sh -c 'curl -s "localhost:8080/codex/{}" | jq -r "\"{}: \" + (.count | tostring) + \" entries\""'

See Also