API Pagination

Pagination strategies for different APIs and implementations.

Offset Pagination

Basic offset pagination — simple but unstable under concurrent writes

# Page 1
curl -s "https://api.example.com/v1/devices?page=1&per_page=50" | jq .

# Page 2
curl -s "https://api.example.com/v1/devices?page=2&per_page=50" | jq .

Loop all pages — offset-based

page=1
per_page=50
while true; do
    response=$(curl -s "https://api.example.com/v1/devices?page=$page&per_page=$per_page")
    count=$(echo "$response" | jq '.items | length')
    echo "$response" | jq -r '.items[] | [.name, .ip] | @tsv'
    (( count < per_page )) && break
    (( page++ ))
done | column -t

Offset pagination skips or duplicates records when items are inserted/deleted between requests. Use cursor pagination for reliable enumeration.

Cursor Pagination

Cursor-based — opaque token from server, stable under writes

cursor=""
while true; do
    if [[ -z "$cursor" ]]; then
        response=$(curl -s "https://api.example.com/v1/devices?limit=50")
    else
        response=$(curl -s "https://api.example.com/v1/devices?cursor=$cursor&limit=50")
    fi
    echo "$response" | jq -r '.items[] | [.name, .status] | @tsv'
    cursor=$(echo "$response" | jq -r '.next_cursor // empty')
    [[ -z "$cursor" ]] && break
done | column -t

Link Header Pagination

Follow RFC 8288 Link headers — HATEOAS-style

url="https://api.example.com/v1/devices?per_page=50"
while [[ -n "$url" ]]; do
    response=$(curl -sD- "$url")
    headers=$(echo "$response" | sed '/^\r$/q')
    body=$(echo "$response" | sed '1,/^\r$/d')
    echo "$body" | jq -r '.[] | .name'
    url=$(echo "$headers" | grep -oP '(?<=<)[^>]+(?=>; rel="next")')
done

ISE ERS Pagination

ISE ERS uses page/size parameters with SearchResult wrapper

page=1
size=100
total=0
while true; do
    response=$(curl -sk -u "$ISE_USER:$ISE_PASS" \
      -H "Accept: application/json" \
      "https://10.50.1.20:9060/ers/config/endpoint?size=$size&page=$page")

    total=$(echo "$response" | jq '.SearchResult.total')
    echo "$response" | jq -r '.SearchResult.resources[] | [.name, .id] | @tsv'

    fetched=$(( page * size ))
    (( fetched >= total )) && break
    (( page++ ))
done | column -t

GitHub API Pagination (gh CLI)

gh CLI handles pagination automatically with --paginate

# List all repos (auto-paginated)
gh api --paginate /user/repos --jq '.[].full_name'

# List all issues across pages
gh api --paginate repos/owner/repo/issues --jq '.[].title'

Manual pagination with gh — when you need control

page=1
while true; do
    response=$(gh api "repos/owner/repo/issues?page=$page&per_page=100")
    count=$(echo "$response" | jq 'length')
    echo "$response" | jq -r '.[] | [.number, .title] | @tsv'
    (( count < 100 )) && break
    (( page++ ))
done

Meraki API Pagination

Meraki uses Link header with startingAfter token

url="https://api.meraki.com/api/v1/organizations/$ORG_ID/devices?perPage=100"
while [[ -n "$url" ]]; do
    response=$(curl -sD- -H "X-Cisco-Meraki-API-Key: $MERAKI_KEY" "$url")
    headers=$(echo "$response" | sed '/^\r$/q')
    body=$(echo "$response" | sed '1,/^\r$/d')
    echo "$body" | jq -r '.[] | [.name, .model, .networkId] | @tsv'
    url=$(echo "$headers" | grep -oP '(?<=<)[^>]+(?=>; rel=next)' || true)
done | column -t

FastAPI Pagination Pattern

Offset pagination in domus-api

from fastapi import Query

@app.get("/v1/devices")
async def list_devices(
    skip: int = Query(0, ge=0, description="Offset"),
    limit: int = Query(50, ge=1, le=100, description="Page size"),
):
    devices = await db.fetch_devices(skip=skip, limit=limit)
    total = await db.count_devices()
    return {
        "items": devices,
        "total": total,
        "skip": skip,
        "limit": limit,
        "has_more": skip + limit < total,
    }