yq Session 05: Infrastructure

Real infrastructure YAML. This session uses your actual repo configs, GitHub Actions workflows, and k8s-style manifests.

Pre-Session State

Can read, filter, edit, merge YAML
Understand multi-document YAML
Know anchors and aliases

Pattern 1: antora.yml Comparison

Compare attributes across your repos:

Exercise 1.1: List all attributes from one repo

yq '.asciidoc.attributes | keys' ~/atelier/_bibliotheca/domus-captures/docs/asciidoc/antora.yml

Exercise 1.2: Extract IPs from attributes

yq -o=json '.asciidoc.attributes' ~/atelier/_bibliotheca/domus-captures/docs/asciidoc/antora.yml |
  jq 'to_entries[] | select(.value | tostring | test("[0-9]+\\.[0-9]+\\.[0-9]+"))'

Exercise 1.3: Compare across repos

for f in ~/atelier/_bibliotheca/domus-*/docs/asciidoc/antora.yml; do
  repo=$(basename "$(dirname "$(dirname "$(dirname "$f")")")")
  count=$(yq '.asciidoc.attributes | length // 0' "$f")
  printf "%-30s %s attributes\n" "$repo" "$count"
done

Exercise 1.4: Find shared attributes

for f in ~/atelier/_bibliotheca/domus-*/docs/asciidoc/antora.yml; do
  repo=$(basename "$(dirname "$(dirname "$(dirname "$f")")")")
  yq -o=json '.asciidoc.attributes // {} | keys' "$f" |
    jq --arg repo "$repo" '.[] | {key: ., repo: $repo}'
done | jq -s 'group_by(.key) | map(select(length > 1)) | .[] | {key: .[0].key, repos: [.[].repo]}'

Pattern 2: GitHub Actions Workflow

Exercise 2.1: Create a test workflow

cat > /tmp/yq-gha.yaml << 'EOF'
name: CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: "20"
      - name: Install
        run: npm ci
      - name: Build
        run: npm run build
  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: npm test
EOF

Exercise 2.2: List all jobs

yq '.jobs | keys' /tmp/yq-gha.yaml

Exercise 2.3: Get all `uses` actions

yq '.jobs[].steps[].uses | select(. != null)' /tmp/yq-gha.yaml

Exercise 2.4: Find steps with `run` commands

yq '.jobs[].steps[] | select(has("run")) | .run' /tmp/yq-gha.yaml

Exercise 2.5: Add a step to a job

yq '.jobs.build.steps += [{"name": "Lint", "run": "npm run lint"}]' /tmp/yq-gha.yaml | yq '.jobs.build.steps[-1]'

Pattern 3: docker-compose

Exercise 3.1: Create a compose file

cat > /tmp/yq-compose.yaml << 'EOF'
version: "3.8"
services:
  wazuh-manager:
    image: wazuh/wazuh-manager:4.7.0
    ports:
      - "1514:1514"
      - "1515:1515"
      - "55000:55000"
    environment:
      WAZUH_MANAGER_IP: 10.50.1.70
    volumes:
      - wazuh-data:/var/ossec/data
    restart: always
  wazuh-indexer:
    image: wazuh/wazuh-indexer:4.7.0
    ports:
      - "9200:9200"
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
    volumes:
      - indexer-data:/var/lib/wazuh-indexer
    restart: always
  grafana:
    image: grafana/grafana:10.2.0
    ports:
      - "3000:3000"
    restart: unless-stopped
volumes:
  wazuh-data:
  indexer-data:
EOF

Exercise 3.2: List all services

yq '.services | keys' /tmp/yq-compose.yaml

Exercise 3.3: Get all exposed ports

yq '.services[].ports[]' /tmp/yq-compose.yaml

Exercise 3.4: Find services with `always` restart

yq '.services | to_entries[] | select(.value.restart == "always") | .key' /tmp/yq-compose.yaml

Exercise 3.5: Update an image tag

yq '.services.grafana.image = "grafana/grafana:11.0.0"' /tmp/yq-compose.yaml | yq '.services.grafana.image'

Pattern 4: Cross-format Pipeline

Exercise 4.1: YAML to JSON to table

yq -o=json '.services | to_entries | map({name: .key, image: .value.image, restart: .value.restart})' /tmp/yq-compose.yaml |
  jq -r '.[] | [.name, .image, .restart] | @tsv' |
  column -t

Exercise 4.2: YAML to CSV

yq -o=json '.services | to_entries | map([.key, .value.image])' /tmp/yq-compose.yaml |
  jq -r '.[] | @csv'

Exercises to Complete

[ ] Find all attributes containing "10.50" across your real antora.yml files
[ ] Extract all uses actions from a real GitHub Actions workflow in one of your repos
[ ] Create a compose file, add a service with yq, remove a volume with del()
[ ] Build a pipeline: yq reads YAML, outputs JSON, jq filters and formats as TSV

Session Log

Timestamp	Notes
Start	<Record when you started>
End	<Record when you finished>

Timestamp

Notes

Start

<Record when you started>

End

<Record when you finished>