Testing & Validation

Testing Strategy

Three layers: async integration tests (pytest), live endpoint validation (bash), and interactive demo (bash). All test against real filesystem data — no mocks, because the filesystem IS the database.

pytest + httpx (48 Tests)

Async integration tests using httpx.AsyncClient with ASGITransport. The lifespan context is triggered manually in the session-scoped fixture, so the DocumentCache loads real files:

# tests/conftest.py
@pytest.fixture(scope="session")
async def client():
    async with lifespan(app):
        transport = ASGITransport(app=app)
        async with AsyncClient(transport=transport, base_url="http://test") as ac:
            yield ac
Test File Tests Coverage

test_health.py

4

Health check, stats, attributes, standards (STD-NNN filtered)

test_pages.py

4

List, filter by category, limit, 404 on invalid path

test_search.py

5

Basic search, scoped search, limit, 422 on missing/short query

test_case_studies.py

7

List all, list by type, 404 unknown type, POST incident/change/RCA, 422 validation

test_projects.py

5

List, POST scaffold, 422 validation, PATCH status, 404 nonexistent slug

test_content_endpoints.py

23

Parametrized 200-check across 19 content endpoints + codex/patterns/education/skills detail

POST tests use cleanup fixtures that delete scaffolded files after the test:

@pytest.fixture
def cleanup_files():
    created: list[Path] = []
    yield created
    for f in created:
        if f.exists():
            f.unlink()

Run: uv run --extra dev pytest tests/ -v — 48 passed in 0.3s.

validate.sh (45 Checks)

Live endpoint validation against a running server. Tests HTTP status codes for all 44 endpoints plus 2 feature checks:

Section Checks What

READ

38

GET every endpoint, verify 200

WRITE

4

POST incident, change, RCA, project — verify 201

PATCH

1

PATCH project status — verify 200

FEATURES

2

STD-NNN template filtered from /standards, cache invalidation (POST → immediate search hit)

Runs demo-cleanup.sh after validation to remove all test artifacts. Cleanup handles both demo and validation filename patterns.

Run: bash validate.sh (requires server on :8080)

demo.sh (Interactive)

Walks through all 44 endpoints with curl | jq formatted output. Pauses between sections for presentation. Creates artifacts that demo-cleanup.sh removes.

Run: bash demo.sh (requires server on :8080)