Universal Patterns

What Universal Patterns Are

Universal patterns are the vendor-agnostic knowledge that applies to every REST API you will ever encounter. Authentication, pagination, error handling, rate limiting, response parsing — these are not ISE concepts or GitHub concepts. They are HTTP concepts that every API implements with minor variations.

netapi extracts these patterns into a shared layer so that learning them once means applying them everywhere.

Why They Matter

Without universal patterns, every new API is a fresh learning curve:

  • ISE uses Basic auth with XML; GitHub uses Bearer tokens with JSON; Cloudflare uses API keys in custom headers.

  • ISE paginates with page and size parameters; GitHub uses Link headers; Cloudflare uses page and per_page.

  • ISE returns errors as XML with a messages array; GitHub returns JSON with a message field; Cloudflare wraps everything in {"success": false, "errors": […​]}.

The surface syntax differs. The underlying structure is always one of a small number of patterns. Learn the patterns, and integrating a new API becomes a fill-in-the-blanks exercise instead of a research project.

Sections

Section What You Learn

Integrating Any API

End-to-end walkthrough: from curl discovery to a working vendor definition. The centerpiece guide.

Authentication

The five auth patterns (Basic, Bearer, API Key, OAuth2, mTLS), when each applies, and how netapi abstracts them.

Response Handling

JSON and XML parsing, field extraction with jq, error detection, and output formatting.

Resilience

Rate limiting, retry strategies, exponential backoff, timeout configuration, and circuit breaking.

Discovery

How to explore an unfamiliar API: probing endpoints, reading OpenAPI specs, and mapping resources to netapi verbs.

Security

TLS verification, certificate pinning, secret storage, and credential rotation.

The Layer Stack Connection

Universal patterns map to the bottom three layers of the netapi layer stack:

Layer Name Universal Patterns Covered

1

Primitives

Authentication helpers, HTTP client abstractions, response parsers. These are the atoms that every vendor-specific implementation composes.

2

Protocols

Transport-layer concerns: REST over HTTPS, RESTCONF, NETCONF. The protocol layer handles TLS, connection pooling, and request serialization.

3

Capabilities

Vendor-agnostic operations: backup, monitoring, compliance checks. Capabilities use primitives and protocols without knowing which vendor is behind them.

Layers 4 (Vendors) and 5 (Workflows) are where vendor-specific knowledge lives. The universal patterns documented here are everything below that line — the foundation that makes multi-vendor automation possible.

Getting Started

If you are integrating a new API, start with Integrating Any API. It walks through the full workflow using a real public API that requires no credentials.

If you are debugging a specific problem (auth failures, pagination issues, rate limiting), go directly to the relevant section above.

Public API Credentials (dsec)

All public API keys are stored in d000/dev/app.env.age and loaded with:

dsource d000 dev/app

Add the following variables via dsec edit d000 dev/app:

Variable Required By How to Get

YOUTUBE_API_KEY

YouTube Data API v3

Google Cloud Console (enable YouTube Data API v3)

REDDIT_CLIENT_ID

Reddit API (OAuth2)

Reddit Apps (create "script" type)

REDDIT_CLIENT_SECRET

Reddit API (OAuth2)

Same as above (shown after app creation)

REDDIT_USER_AGENT

Reddit API

Set to netapi/0.2.0 (Reddit requires a descriptive User-Agent)

GOOGLE_SEARCH_API_KEY

Google Custom Search

Google Cloud Console (enable Custom Search API)

GOOGLE_SEARCH_CX

Google Custom Search

Programmable Search Engine (create engine, copy CX ID)

STACKEXCHANGE_API_KEY

StackExchange (optional)

StackApps (optional — increases rate limit from 300 to 10,000/day)

OMDB_API_KEY

OMDB (movies/TV)

OMDB API (free tier: 1,000/day)

SPOTIFY_CLIENT_ID

Spotify Web API

Spotify Developer Dashboard

SPOTIFY_CLIENT_SECRET

Spotify Web API

Same as above

TODOIST_API_TOKEN

Todoist

Todoist App Console

No credentials needed for: Hacker News, Wikipedia, PyPI.

To clean up loaded credentials:

dsunsource

Related