ETL Pipeline Training

Master data pipelines with CLI tools. Extract from APIs, transform with jq/awk/sed, load to files and reports.

What is ETL?

  • Extract - Get data (curl, cat, netapi, kubectl)

  • Transform - Process data (jq, awk, sed, grep)

  • Load - Output data (file, database, API, report)

CLI pattern: curl API | jq transform | output

Sessions (Curriculum)

Session Focus Duration Status

01

Pipes, tee, process substitution, xargs

45 min

[ ] Planned

02

jq extraction, @csv output, awk formatting

45 min

[ ] Planned

03

grep filtering, sed transforms, awk aggregation

60 min

[ ] Planned

04

curl+jq, data enrichment, report generation

60 min

[ ] Planned

05

ISE, k8s, network, SIEM routing patterns

60 min

[ ] Planned

Drills (Practice)

Drill Focus Difficulty

01

Pipes, tee, xargs, process substitution

Beginner

02

jq @csv, awk formatting, transforms

Beginner

03

grep filtering, sed transforms, awk aggregation

Intermediate

04

curl+jq, data enrichment, report generation

Advanced

05

ISE, k8s, network, Monad-style routing

Boss Level

Quick Start

# Run the drill menu
bash ~/atelier/_bibliotheca/domus-captures/docs/modules/ROOT/examples/etl-drills/00-run-drill.sh

# Or run a specific drill
bash ~/atelier/_bibliotheca/domus-captures/docs/modules/ROOT/examples/etl-drills/00-run-drill.sh 01

Related