Monad Pipeline Mastery

Monad is an ETL (Extract-Transform-Load) pipeline platform that sits between log sources and SIEMs. Master transforms, routing logic, and cost optimization to reduce Sentinel ingestion by 50-70%.

Why This Matters

Management is betting on Monad for QRadar → Sentinel migration. Your role:

  • Design transform logic for ISE, FTD, ASA log routing

  • Reduce Sentinel costs via intelligent filtering

  • Maintain compliance with S3 archival

  • Enable MITRE ATT&CK tagging for threat hunting

Quick Reference

Environment Setup

# Load credentials (adjust path for your setup)
dsource d000 lab/app

# Verify
echo "Org: $MONAD_ORG_ID"
monad_summary

Essential Commands

# List pipelines
monad_pipelines | jq '.pipelines[] | {name, enabled}'

# List available transform types
monad_transform_types | jq '.[].type_id'

# View pipeline with transforms
monad_pipeline_v2 <pipeline-id> | jq '.nodes'

# Check syslog endpoint
echo "${PIPELINE_ID}.l4.monad.com:6514"

Curriculum

Module Description Priority

Architecture

Pipeline structure, nodes, edges, data flow

Core

Transform Types

All 12 transform types with use cases

Core

jq Transforms

Complex routing with jq expressions

Critical

GJSON Paths

UI-based transforms using GJSON queries

Core

Routing Patterns

CHLA-specific log routing for Sentinel/S3

Critical

Hands-On Lab

Practical exercises with synthetic logs

Practice

Architecture Overview

                    ┌─────────────┐
                    │  Log Source │
                    │  (ISE, FTD) │
                    └──────┬──────┘
                           │ Syslog/API
                           ▼
┌──────────────────────────────────────────────────────────┐
│                    MONAD PIPELINE                        │
│  ┌─────────┐    ┌────────────┐    ┌─────────────────┐   │
│  │  INPUT  │───▶│ TRANSFORMS │───▶│     OUTPUT      │   │
│  │ Syslog  │    │  jq/drop/  │    │ Sentinel or S3  │   │
│  │  HTTP   │    │  rename    │    │                 │   │
│  └─────────┘    └────────────┘    └─────────────────┘   │
└──────────────────────────────────────────────────────────┘
                           │
              ┌────────────┴────────────┐
              ▼                         ▼
       ┌─────────────┐          ┌─────────────┐
       │  Sentinel   │          │     S3      │
       │  (Critical) │          │  (Archive)  │
       └─────────────┘          └─────────────┘

Decision Matrix: What Goes Where

Log Type Examples Destination MITRE

Auth Failures

ISE RADIUS reject, VPN auth fail

Sentinel

T1078

Policy Violations

ISE posture fail, FTD deny

Sentinel

T1190

Admin Actions

TACACS commands, config changes

Sentinel

T1059

Threat Events

FTD intrusion, malware detect

Sentinel

T1071

Successful Auth

ISE pass, VPN connect

S3 Archive

-

Health Checks

Keepalives, heartbeats

S3 Archive

-

Flow Logs

FTD allow, permit

S3 Archive

-

Transform Types Quick Reference

Type Use Case Example

drop_record_where_value_eq

Filter out low-value logs

Drop where severity="info"

add

Tag logs for routing

Add destination="sentinel"

rename_key

Normalize field names

src_ip → source.ip

jq

Complex routing logic

MITRE ATT&CK tagging

convert_timestamp

Timestamp normalization

epoch → ISO8601

flatten

Flatten nested JSON

Normalize deep structures

Daily Workflow

1. Check Pipeline Health

monad_summary
monad_logs_pipelines | jq '.logs | length'

2. View Transform Logic

monad_pipeline_v2 <id> | jq '.nodes[] | select(.component_type=="transform")'

3. Test with Synthetic Logs

# Send test syslog message
./examples/monad/testing/synthetic-logs.sh <pipeline-id> 1

4. Monitor Output

# Check if logs are flowing
monad_logs_pipeline <id> | jq '.logs[-5:]'

Resources

Related Skills