Integrating Any API

This guide walks through integrating a real public API with netapi, start to finish. The teaching example is JSONPlaceholder — a free REST API with no auth required, which makes it ideal for learning the workflow before applying it to production APIs that need credentials.

The process has seven steps. By the end, you will have a working vendor definition and CLI commands for a new API.

Step 1: Discover

Before writing any configuration, probe the API manually with curl. The goal is to answer four questions:

  • What is the base URL?

  • What authentication does it require?

  • What resources does it expose?

  • How does it paginate?

# Probe the base URL — check if it returns something useful
curl -s https://jsonplaceholder.typicode.com | jq 'keys' 2>/dev/null || echo "Not JSON root"

# List posts (the primary resource)
curl -s https://jsonplaceholder.typicode.com/posts | jq 'length'
# => 100

# Get a single post
curl -s https://jsonplaceholder.typicode.com/posts/1 | jq .
# => { "userId": 1, "id": 1, "title": "...", "body": "..." }

# Check response headers for pagination and rate limit clues
curl -sI https://jsonplaceholder.typicode.com/posts

From this discovery:

  • Base URL: jsonplaceholder.typicode.com

  • Auth: None (public API)

  • Resources: /posts, /comments, /users, /todos, /albums, /photos

  • Pagination: None — the API returns all records in one response

For production APIs, you would also look for:

  • Link headers (GitHub-style pagination)

  • X-RateLimit-* headers

  • Required auth headers (check for 401 Unauthorized without credentials)

  • API documentation or OpenAPI specs

Step 2: Authenticate

JSONPlaceholder needs no authentication, but the pattern matters. When you encounter a real API, map its auth scheme to one of netapi’s universal types:

API Says netapi Auth Type What You Configure

"Pass a token in the Authorization: Bearer header"

bearer

token_env: MY_TOKEN

"Use HTTP Basic Authentication"

basic

username_env + password_env

"Send your API key in the X-API-Key header"

api-key

key_env + header

"Register an OAuth2 application"

oauth2

client_id_env + client_secret_env + token_url

"Use mutual TLS with a client certificate"

mtls

cert_path + key_path + ca_path

For JSONPlaceholder, we skip auth entirely. The vendor definition will omit the auth block (or set type: none).

Step 3: First Call

Make the simplest possible request and parse the response:

# Fetch a single resource
curl -s https://jsonplaceholder.typicode.com/posts/1 | jq .
{
  "userId": 1,
  "id": 1,
  "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
  "body": "quia et suscipit\nsuscipit recusandae..."
}

Identify the shape:

  • ID field: id (integer)

  • Foreign keys: userId links to /users

  • Content fields: title, body

# Extract just the fields you care about
curl -s https://jsonplaceholder.typicode.com/posts/1 \
  | jq '{id, title, userId}'

# List resources and extract a summary
curl -s https://jsonplaceholder.typicode.com/posts \
  | jq '.[:5] | .[] | {id, title}'

Step 4: CRUD Lifecycle

Work through create, read, update, delete — verifying at each step.

Create

curl -s -X POST https://jsonplaceholder.typicode.com/posts \
  -H "Content-Type: application/json" \
  -d '{
    "title": "netapi test post",
    "body": "Testing CRUD lifecycle",
    "userId": 1
  }' | jq .
{
  "title": "netapi test post",
  "body": "Testing CRUD lifecycle",
  "userId": 1,
  "id": 101
}
JSONPlaceholder fakes mutations — it returns the expected response but does not persist changes. This is fine for learning the workflow.

Read

# Read the resource we just "created"
curl -s https://jsonplaceholder.typicode.com/posts/101 | jq .

Update

# Full replacement (PUT)
curl -s -X PUT https://jsonplaceholder.typicode.com/posts/1 \
  -H "Content-Type: application/json" \
  -d '{
    "id": 1,
    "title": "Updated title",
    "body": "Updated body",
    "userId": 1
  }' | jq .

# Partial update (PATCH)
curl -s -X PATCH https://jsonplaceholder.typicode.com/posts/1 \
  -H "Content-Type: application/json" \
  -d '{"title": "Only the title changed"}' | jq .

Delete

curl -s -X DELETE https://jsonplaceholder.typicode.com/posts/1 \
  -w "\nHTTP Status: %{http_code}\n"
# => HTTP Status: 200 (empty body on success)

Step 5: Handle Edges

Production APIs throw errors, paginate, and rate-limit. Test these scenarios during discovery:

Error Responses

# Non-existent resource — expect 404
curl -s -w "\nHTTP %{http_code}" https://jsonplaceholder.typicode.com/posts/9999
# => {} with HTTP 404

# Invalid endpoint — expect 404
curl -s -w "\nHTTP %{http_code}" https://jsonplaceholder.typicode.com/nonexistent
# => HTTP 404

# Malformed request body
curl -s -X POST https://jsonplaceholder.typicode.com/posts \
  -H "Content-Type: application/json" \
  -d 'not json' -w "\nHTTP %{http_code}"

Pagination

JSONPlaceholder supports query parameter filtering but not true pagination. For APIs that do paginate, test the mechanics:

# Simulated: JSONPlaceholder supports _start and _limit
curl -s "https://jsonplaceholder.typicode.com/posts?_start=0&_limit=5" | jq 'length'
# => 5

curl -s "https://jsonplaceholder.typicode.com/posts?_start=5&_limit=5" | jq 'length'
# => 5

Rate Limits

# Check for rate limit headers
curl -sI https://jsonplaceholder.typicode.com/posts \
  | grep -i "rate-limit"
# JSONPlaceholder has no rate limits — production APIs will

For APIs with rate limits, note:

  • Which header reports remaining requests

  • Which header reports the reset time

  • Whether they use 429 status codes

Step 6: Automate

Now translate your curl exploration into a vendor definition:

# ~/.config/netapi/vendors/jsonplaceholder.yml
name: jsonplaceholder
base_url: https://jsonplaceholder.typicode.com

auth:
  type: none

pagination:
  type: offset
  params:
    offset: _start
    limit: _limit

rate_limit:
  strategy: none

resources:
  post:
    list: GET /posts
    get: GET /posts/{id}
    create: POST /posts
    update: PUT /posts/{id}
    delete: DELETE /posts/{id}
  comment:
    list: GET /posts/{post_id}/comments
    get: GET /comments/{id}
  user:
    list: GET /users
    get: GET /users/{id}
  todo:
    list: GET /todos
    get: GET /todos/{id}

Validate and test:

# Validate the YAML structure
netapi vendor validate jsonplaceholder

# Test with a live request
netapi vendor validate jsonplaceholder --live

# Use it
netapi jsonplaceholder post list --format table
netapi jsonplaceholder post get 1 --format json
netapi jsonplaceholder user list --format json | jq '.[].name'
netapi jsonplaceholder todo list --format json | jq '[.[] | select(.completed == false)] | length'

The same workflow applies to any production API. Replace jsonplaceholder with github, cloudflare, or your internal service, fill in the auth and pagination details, and the CLI commands work identically.

Step 7: Document

Once the vendor definition works, create documentation so others can use it. A vendor’s documentation should cover:

  1. Prerequisites — What credentials are needed, where to get them

  2. Available resources — What you can query

  3. Common workflows — The 3-5 things people actually do with this API

  4. Gotchas — Rate limits, pagination quirks, required headers

For netapi’s built-in vendors, this documentation lives in the CLI reference under cli/<vendor>/. For user-defined vendors, keep a README alongside the YAML definition or contribute it upstream.

Applying This to Production APIs

The JSONPlaceholder walkthrough covers the mechanics. Here is how the steps differ for real APIs:

Step Production Considerations

Discover

Start with the vendor’s API documentation. Look for OpenAPI/Swagger specs — netapi can import these directly.

Authenticate

Store credentials in environment variables, never in the YAML. Use dsec for secret management: dsec get github.token.

First call

Expect TLS certificate verification issues with self-signed certs. Use --verbose to debug.

CRUD

Test against a staging environment. Use --dry-run for destructive operations until you trust the definition.

Edges

Map the API’s specific error format. Some return {"error": "message"}, others {"errors": [{"detail": "…​"}]}.

Automate

Start with read-only verbs (list, get, count). Add write verbs after validation.

Document

Include the output of netapi vendor show <name> so others can verify their setup matches.

Related