jq — API Patterns

curl | jq — The Basic Pattern

Fetch and pretty-print JSON from an API
curl -s https://jsonplaceholder.typicode.com/users/1 | jq '.'
# Output:
# {
#   "id": 1,
#   "name": "Leanne Graham",
#   "username": "Bret",
#   "email": "Sincere@april.biz",
#   ...
# }

curl -s (silent) suppresses progress bars. Always use -s when piping to jq.

Extract specific fields
curl -s https://jsonplaceholder.typicode.com/users | \
    jq '.[] | {name: .name, email: .email, city: .address.city}'
# Output:
# {"name": "Leanne Graham", "email": "Sincere@april.biz", "city": "Gwenborough"}
# {"name": "Ervin Howell", "email": "Shanna@melissa.tv", "city": "Wisokyburgh"}
# ...
Filter API results
curl -s https://jsonplaceholder.typicode.com/todos | \
    jq '[.[] | select(.completed == true)] | length'
# Output: 90  (completed todos out of 200)

--arg for Dynamic Queries

Pass shell variables safely into jq
USER_ID=3
curl -s https://jsonplaceholder.typicode.com/posts | \
    jq --argjson uid "$USER_ID" '[.[] | select(.userId == $uid)] | length'
# Output: 10  (posts by user 3)

--argjson for numbers and JSON values. --arg for strings. Never interpolate shell variables directly into jq expressions — it breaks on special characters and is an injection vector.

Multiple variables
START=1
END=5
curl -s https://jsonplaceholder.typicode.com/posts | \
    jq --argjson s "$START" --argjson e "$END" '[.[] | select(.id >= $s and .id <= $e)]'
# Output: posts 1 through 5

--slurpfile for Multi-Source Joins

Join two API responses
# Fetch users into a temp file
curl -s https://jsonplaceholder.typicode.com/users > /tmp/users.json

# Fetch posts and enrich with user names
curl -s https://jsonplaceholder.typicode.com/posts | \
    jq --slurpfile users /tmp/users.json '
    .[:5] | map(. + {
        author: ($users[0][] | select(.id == .userId) | .name) // "unknown"
    }) | .[] | {title: .title, author: .author}'

--slurpfile var file loads a JSON file as a jq variable. The value is always an array (even if the file contains a single object), so access with $var[0].

Error Handling

Handle HTTP errors gracefully
response=$(curl -s -w "\n%\\{http_code}" https://jsonplaceholder.typicode.com/users/1)
http_code=$(echo "$response" | tail -1)
body=$(echo "$response" | sed '$d')

if [ "$http_code" -eq 200 ]; then
    echo "$body" | jq -r '.name'
else
    echo "Error: HTTP $http_code" >&2
fi
# Output: Leanne Graham

curl -w "\n%\{http_code}" appends the HTTP status code after the response body. Split them to handle errors before jq parsing.

jq try-catch for malformed responses
echo 'not json at all' | jq -r 'try .name catch "PARSE_ERROR"'
# Output: PARSE_ERROR
Default values for missing fields
curl -s https://jsonplaceholder.typicode.com/users/1 | \
    jq '{
        name: .name,
        phone: (.phone // "N/A"),
        department: (.department // "unassigned")
    }'
# Output:
# {
#   "name": "Leanne Graham",
#   "phone": "1-770-736-8031 x56442",
#   "department": "unassigned"
# }

// (alternative operator) provides a fallback when the left side is null or false.

Headers and Authentication

Bearer token authentication
TOKEN="your-api-token"
curl -s -H "Authorization: Bearer $TOKEN" \
    -H "Accept: application/json" \
    https://api.example.com/v1/resources | jq '.data[:5]'
API key in header
curl -s -H "X-API-Key: $API_KEY" \
    https://api.example.com/v1/status | jq '{status: .status, version: .version}'
POST with JSON body
curl -s -X POST https://jsonplaceholder.typicode.com/posts \
    -H "Content-Type: application/json" \
    -d "$(jq -n --arg title "Test Post" --arg body "Content here" \
        '{title: $title, body: $body, userId: 1}')" | jq '.'
# Output:
# {
#   "title": "Test Post",
#   "body": "Content here",
#   "userId": 1,
#   "id": 101
# }

Using jq -n to construct the POST body ensures proper JSON encoding — no manual escaping needed.

Pagination Patterns

Offset-based pagination loop
page=1
per_page=10
while true; do
    response=$(curl -s "https://api.example.com/items?page=$page&per_page=$per_page")
    count=$(echo "$response" | jq '.items | length')
    echo "$response" | jq -c '.items[]'
    [ "$count" -lt "$per_page" ] && break
    page=$((page + 1))
done | jq -s '.'
# Collects all pages into a single array
Link header pagination (GitHub API style)
url="https://api.github.com/repos/jqlang/jq/issues?per_page=5&state=open"
while [ -n "$url" ]; do
    response=$(curl -sI "$url")
    curl -s "$url" | jq -c '.[]'
    url=$(echo "$response" | grep -i '^link:' | grep -o '<[^>]*>; rel="next"' | \
        sed 's/<\(.*\)>; rel="next"/\1/')
done | jq -s 'length'
# Counts all open issues across pages

Practical: API Exploration Workflow

Discover API structure
# Step 1: What keys exist at top level?
curl -s https://jsonplaceholder.typicode.com/users/1 | jq 'keys'

# Step 2: What types are the values?
curl -s https://jsonplaceholder.typicode.com/users/1 | \
    jq 'to_entries | map({key: .key, type: (.value | type)})'

# Step 3: How deep is the nesting?
curl -s https://jsonplaceholder.typicode.com/users/1 | \
    jq '[path(..) | length] | max'
# Output: 3  (deepest path is 3 levels)

# Step 4: Extract the structure you need
curl -s https://jsonplaceholder.typicode.com/users/1 | \
    jq '{name, email, geo: .address.geo}'

This four-step pattern works for any unfamiliar API: keys, types, depth, then targeted extraction.

Saving and Loading Results

Save intermediate results for reuse
# Fetch once, query many times
curl -s https://jsonplaceholder.typicode.com/users > /tmp/api-users.json

# Query 1: emails
jq -r '.[].email' /tmp/api-users.json

# Query 2: users in specific city
jq '[.[] | select(.address.city == "Gwenborough")]' /tmp/api-users.json

# Query 3: CSV export
jq -r '.[] | [.name, .email, .address.city] | @csv' /tmp/api-users.json

Fetch once into a file, query repeatedly. Saves API calls and avoids rate limiting.