API Exploration with curl

When there is no OpenAPI spec — or the spec is incomplete — curl is the ground truth. Verbose mode shows you exactly what the server sends back: headers, status codes, redirects, content types. This page covers a systematic approach to mapping an unknown API.

The discovery pattern

Start broad, narrow down. Every probe teaches you something.

# Step 1: What is at the base URL?
curl -sI https://api.example.com/

# Step 2: Check common API paths
for path in /api /v1 /v2 /api/v1 /api/v2 /rest /graphql; do
  code=$(curl -so /dev/null -w '%{http_code}' "https://api.example.com${path}")
  echo "${code} ${path}"
done

# Step 3: When you find a live path, inspect it
curl -sv https://api.example.com/api/v1 2>&1 | head -50

A 200 or 401 means the path exists. A 301/302 means it exists but redirects — follow it. A 404 means nothing is there. A 403 means something is there but you lack access.

Verbose mode for debugging

curl -v prints the full request and response, including TLS negotiation, request headers, and response headers.

curl -v https://api.example.com/api/v1/users 2>&1

The output has three sections:

>

Lines you sent (request headers)
<

Lines the server sent (response headers)
*

curl’s internal notes (TLS, connection reuse, timing)

For cleaner output when you only care about headers:

# Response headers only (-s suppresses progress, -D - dumps headers to stdout)
curl -sD - -o /dev/null https://api.example.com/api/v1/users

Reading response headers

Response headers reveal API behavior before you parse a single byte of the body.

curl -sI https://api.example.com/api/v1/users \
  -H "Authorization: Bearer $TOKEN"

What to look for:

Header What it tells you

Content-Type

Response format (JSON, XML, HTML) — confirms you hit an API endpoint, not a web page

WWW-Authenticate

Authentication scheme the server expects (on 401 responses)

X-RateLimit-*

Rate limiting policy — how fast you can call

X-Request-Id

Request tracing — useful for support tickets

Allow

Which HTTP methods are permitted (on 405 responses)

Link

Pagination URLs (GitHub-style)

X-Total-Count

Total items available (some APIs include this)

Vary

Which request headers affect caching — hints at content negotiation

X-Powered-By

Server framework (Django, Express, Spring) — helps predict API patterns

Follow redirects

Some APIs redirect from /api to /api/v2 or from HTTP to HTTPS. Without -L, curl stops at the redirect and shows a 3xx response.

# Follow redirects automatically
curl -sL https://api.example.com/api | jq 'keys'

# See the redirect chain
curl -sIL https://api.example.com/api 2>&1 | grep -E 'HTTP/|Location:'

Cookie handling

Some APIs (particularly web-app APIs not designed as public APIs) use session cookies instead of tokens.

# Login and save cookies
curl -s -c cookies.txt -X POST https://app.example.com/login \
  -d '{"username": "admin", "password": "secret"}'

# Use saved cookies for subsequent requests
curl -s -b cookies.txt https://app.example.com/api/data

# Combined: save and send cookies (maintains session across requests)
curl -s -b cookies.txt -c cookies.txt https://app.example.com/api/data
-c file

Save cookies the server sends (Set-Cookie headers)
-b file

Send cookies from a file with the request
-b "name=value"

Send a specific cookie inline

HEAD requests for metadata

HEAD returns headers without a body. Use it to check if a resource exists or to read metadata without downloading content.

# Check if a resource exists (fast)
curl -sI -o /dev/null -w '%{http_code}' \
  https://api.example.com/api/v1/users/12345

# Check content type and size without downloading
curl -sI https://api.example.com/api/v1/reports/export \
  | grep -iE 'content-type|content-length'

Not all APIs support HEAD. If you get 405 Method Not Allowed, the endpoint only accepts GET/POST.

Content negotiation

APIs sometimes serve different formats based on the Accept header.

# Request JSON explicitly
curl -s https://api.example.com/api/v1/users \
  -H "Accept: application/json"

# Request XML
curl -s https://api.example.com/api/v1/users \
  -H "Accept: application/xml"

# See what the server prefers when you do not specify
curl -sI https://api.example.com/api/v1/users \
  | grep -i content-type

Cisco ISE ERS requires explicit Accept headers and returns 406 Not Acceptable without them.

Building a mental model of an unknown API

A systematic exploration builds understanding in layers:

#!/usr/bin/env bash
# API reconnaissance script
# Usage: ./api-recon.sh https://api.example.com

BASE_URL="${1:?Usage: $0 <base-url>}"

echo "=== Base URL ==="
curl -sI "$BASE_URL" | grep -iE 'HTTP/|content-type|server|x-powered'

echo ""
echo "=== Common paths ==="
for path in / /api /v1 /v2 /api/v1 /api/v2 /rest /graphql \
            /health /status /version /info \
            /openapi.json /swagger.json /v3/api-docs /docs; do
  code=$(curl -so /dev/null -w '%{http_code}' "${BASE_URL}${path}")
  if [[ "$code" != "404" ]]; then
    printf "  %s  %s\n" "$code" "$path"
  fi
done

echo ""
echo "=== Auth probe (unauthenticated) ==="
response=$(curl -sD - -o /dev/null "${BASE_URL}/api/v1" 2>/dev/null)
echo "$response" | grep -iE 'www-authenticate|x-ratelimit|content-type'

echo ""
echo "=== Rate limit headers ==="
curl -sI "${BASE_URL}/" | grep -iE 'x-ratelimit|retry-after|ratelimit'

After running this:

  1. You know which paths exist (200, 401, 403)

  2. You know the authentication scheme (WWW-Authenticate header)

  3. You know the response format (Content-Type)

  4. You know if a machine-readable spec exists

  5. You know the rate limit policy

From here, authenticate and start probing individual endpoints. Each response teaches you the data model. Within 20-30 requests you can map most APIs well enough to start building automation.

Timing and performance

curl can report timing for each phase of the request:

curl -s -o /dev/null -w "\
  DNS:       %{time_namelookup}s\n\
  Connect:   %{time_connect}s\n\
  TLS:       %{time_appconnect}s\n\
  Start:     %{time_starttransfer}s\n\
  Total:     %{time_total}s\n\
  HTTP Code: %{http_code}\n" \
  "$API_URL/endpoint"

This is valuable when debugging latency. If time_namelookup is slow, the problem is DNS. If time_appconnect is slow, TLS negotiation is the bottleneck. If time_starttransfer is slow, the server is thinking.