STD-027: Script Extraction

Every script longer than 5 lines in a standalone project partial MUST be extracted to the project’s scripts/ directory. Bash and Python scripts are referenced via pasteable run commands. PowerShell scripts use a deployer heredoc pattern with include:: to transfer the script to the target machine. Scripts are executable artifacts, not documentation fragments.

Principles

  1. Scripts are executable artifacts. A script embedded in a partial cannot be chmod +x, tested with shellcheck, run with strace, or versioned independently. Extraction makes it a first-class citizen.

  2. One source, three roles. A script in scripts/ serves as an executable (bash script.sh), a sourceable function (source script.sh), and documentation content (include::{scriptsdir}/script.sh[]). Inline duplication breaks all three.

  3. The run command is the documentation. The book shows how to execute. The source lives in the file. Readers who want the source open the file — the book does not duplicate it.

  4. Aliases are not portable. Shell aliases (build-adoc, d001) do not exist in non-interactive bash. Scripts MUST resolve the real path via environment variables with sensible defaults.

Requirements

  1. Script blocks exceeding 5 lines in any partials/*.adoc file MUST be extracted to scripts/ as standalone files (.sh, .ps1, .py).

  2. Every standalone project (data/d00x/projects/) MUST have a scripts/ directory. Empty is acceptable; absent is not.

  3. Scripts SHOULD be named by action: setup-postgres.sh, dcr-inventory.ps1, build-enhanced-report.py — not script1.sh or utils.ps1.

  4. Bash/Python: the partial MUST reference the script with a pasteable run command only. No include:: of the script source. The run command is the interface — the source lives in the file.

    \[source,bash]
\----
bash $(find data/d000/projects/<slug>/scripts -name 'script-name.sh')
\----

  5. PowerShell: the partial MUST use the deployer heredoc pattern — @'…​'@ | Set-Content with include::{scriptsdir}/<name>.ps1[] pulling the script content at build time. See PowerShell — Deployer Pattern.

  6. The run command MUST use $(find) for paths. No $ prompt glyph.

  7. Bash scripts MUST be executable (chmod +x) and MUST begin with #!/usr/bin/env bash and set -euo pipefail.

  8. PowerShell scripts MUST be pure module code — no deployer wrapper inside the .ps1 file.

  9. Scripts that call build-adoc or any shell alias MUST resolve the real path via an environment variable with a default:

    BUILD_ADOC="${BUILD_ADOC:-$HOME/atelier/_bibliotheca/domus-asciidoc-build/bin/build-adoc.sh}"

  10. Build function scripts (e.g., build-<project>.sh) MUST support both direct execution and sourcing:

    # If executed directly (not sourced), run the function
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
  build-project-name "$@"
fi

The Pattern

Before (non-compliant)

Script content embedded directly in a partial:

==== Postgres Setup

\[source,bash]
\----
docker run -d \
  --name domus_postgres \
  -e POSTGRES_USER=domus \
  ...
  postgres:16-alpine

psql -h localhost -U domus -d domus_index -f sql/001-create-tables.sql
\----

After (compliant)

Script extracted to scripts/setup-postgres.sh, partial shows only the run command:

==== Postgres Setup

\[source,bash]
\----
bash $(find data/d000/projects/domus-index/scripts -name 'setup-postgres.sh')
\----

Build Function Pattern

Every project SHOULD have a build-<project>.sh in scripts/:

bash data/d000/projects/<slug>/scripts/build-<slug>.sh
bash data/d000/projects/<slug>/scripts/build-<slug>.sh html catppuccin
bash data/d000/projects/<slug>/scripts/build-<slug>.sh pdf navy-executive
bash data/d000/projects/<slug>/scripts/build-<slug>.sh both

Or sourced for interactive use:

source data/d000/projects/<slug>/scripts/build-<slug>.sh
build-<slug> html obsidian-amber

Starter Template

Copy this skeleton for any extracted script. Fill in <slug> and the body:

Three rules that have caused repeated terminal crashes and session loss:

  1. No set -euo pipefail at top level in sourceable scripts. When you source a file, strict mode applies to your entire shell — any non-zero return (a failed glob, a missing command) kills the terminal. Strict mode belongs inside the function body or is omitted entirely for build functions.

  2. No bare BASH_SOURCE — zsh does not define it. Use the dual-guard pattern below.

  3. Default verb is open (build + Firefox), not build. The common case is seeing the output.

 
#!/usr/bin/env bash
# <slug> — reusable build function
#
# Usage:
#   bash scripts/build-<slug>.sh                           # build + open (default)
#   bash scripts/build-<slug>.sh build html catppuccin     # build only
#   bash scripts/build-<slug>.sh html obsidian-amber       # build + open
#   bash scripts/build-<slug>.sh help                      # list variants
#
# Source for interactive use (bash OR zsh):
#   source scripts/build-<slug>.sh
#   build-<slug> html royal
#
# NOTE: no top-level set -euo pipefail — sourcing would kill the terminal.

# Resolve build-adoc — alias not available in non-interactive bash
BUILD_ADOC="${BUILD_ADOC:-$HOME/atelier/_bibliotheca/domus-asciidoc-build/bin/build-adoc.sh}"

build-<slug>() {
  local proj="$HOME/atelier/_bibliotheca/domus-captures/data/d000/projects/<slug>"
  local doc="$proj/<slug>.adoc"

  # Default verb: open (build + Firefox)
  local verb="open"
  case "${1:-}" in
    build|open) verb="$1"; shift ;;
    help)       verb="help"; shift ;;
  esac

  local format="${1:-html}"
  local variant="${2:-light-cyan}"
  local theme="${2:-navy-executive}"

  # Render diagrams before build
  local img="$proj/images"
  mkdir -p "$img" "$proj/output"
  for f in "$proj"/diagrams/*.d2; do
    [[ -f "$f" ]] && d2 "$f" "$img/$(basename "$f" .d2).svg" 2>/dev/null
  done
  for f in "$proj"/diagrams/*.dot; do
    [[ -f "$f" ]] && dot -Tsvg "$f" -o "$img/$(basename "$f" .dot).svg"
  done

  case "$format" in
    html)
      "$BUILD_ADOC" "$doc" html --variant "$variant" -a env-d000 || return
      printf '[OK] HTML built: %s/output/<slug>.html\n' "$proj"
      [[ "$verb" == "open" ]] && firefox "$proj/output/<slug>.html" >/dev/null 2>&1 &
      ;;
    pdf)
      "$BUILD_ADOC" "$doc" pdf --theme "$theme" -a env-d000 || return
      printf '[OK] PDF built: %s/output/<slug>.pdf\n' "$proj"
      [[ "$verb" == "open" ]] && firefox "$proj/output/<slug>.pdf" >/dev/null 2>&1 &
      ;;
    both)
      "$BUILD_ADOC" "$doc" html --variant "$variant" -a env-d000 || return
      "$BUILD_ADOC" "$doc" pdf --theme "$theme" -a env-d000 || return
      printf '[OK] HTML + PDF built: %s/output/\n' "$proj"
      [[ "$verb" == "open" ]] && firefox "$proj/output/<slug>.html" >/dev/null 2>&1 &
      ;;
    help|*)
      printf 'Usage: build-<slug> [build|open] [html|pdf|both] [variant/theme]\n'
      # ... variant/theme list ...
      return 0
      ;;
  esac
}

# If executed directly (not sourced), run the function
# Compatible with both bash and zsh
if [[ -n "${BASH_SOURCE+x}" ]]; then
  [[ "${BASH_SOURCE[0]}" == "${0}" ]] && build-<slug> "$@"
elif [[ -n "${ZSH_EVAL_CONTEXT+x}" ]]; then
  [[ "$ZSH_EVAL_CONTEXT" != *:file* ]] && build-<slug> "$@"
fi

Migrating an Existing Project

Turning a non-compliant project (inline commands in README.adoc / partials) into a compliant one — a five-minute conversion for a typical project:

  1. Find the offenders — inline [source,bash] blocks over 5 lines in partials/*.adoc and README.adoc.

  2. For each, create scripts/<action>.sh from the Starter Template and move the body in.

  3. chmod +x scripts/*.sh.

  4. Replace each inline block with its run command — bash $(find data/d000/projects/<slug>/scripts -name '<action>.sh'), or source …​ for build functions.

  5. Run the Compliance checks below; all must pass before you consider the project migrated.

Exceptions

  1. Educational examples (glob-mastery, regex-mastery, vim-adventures): inline code blocks that exist as teaching content — not operational scripts — MAY remain inline. The test: would you execute this as a standalone file? If no, it stays inline.

  2. One-liners (5 lines or fewer): short commands like psql -c "select version();" or Get-AzContext | Format-List MAY remain inline. They are documentation, not scripts.

  3. README.adoc quick-deploy blocks: the one-paste inline command in README MAY remain as a convenience, provided the same logic also exists as a script in scripts/.

  4. PowerShell ad-hoc queries: single Invoke-AzOperationalInsightsQuery calls with inline KQL MAY remain inline if they are one-off exploration. Queries that recur across sessions MUST be added to the $Queries hashtable in the inventory module.

Compliance

Check Method Pass Criterion

No inline scripts >5 lines

grep -c '\[source,bash\]|\[source,powershell\]' partials/*.adoc cross-referenced against extraction

Every code block >5 lines is extracted or uses include::

scripts/ directory exists

test -d scripts/

Directory present in every standalone project

Bash scripts are executable

find scripts/ -name '*.sh' ! -perm -u+x

Zero results

Alias resolution

grep -r 'build-adoc' scripts/

Zero bare build-adoc calls — all use "$BUILD_ADOC"

No bash/python source in document

grep -rE 'include::.(\.sh|\.py)\[\]' partials/.adoc

Zero include:: of .sh/.py files — only run commands

PowerShell uses deployer pattern

grep -rE 'include::.\.ps1\[\]' partials/.adoc

Every .ps1 include is inside an @'…​'@ | Set-Content deployer block

PowerShell scripts are pure modules

grep -l 'Set-Content' scripts/*.ps1

Zero results — no deployer wrapper inside .ps1 files

Shebang + strict mode (bash)

head -2 scripts/*.sh

Every .sh starts with #!/usr/bin/env bash and set -euo pipefail

d001 scripts encrypted

find scripts/ -name '.ps1' -o -name '.sh' | while read f; do test -f "$f.age" || echo "MISSING: $f.age"; done

Every script in data/d001/ has a corresponding .age file

Platform Adaptation

This standard applies to all script languages. The separation is universal: the script is the artifact, the document shows how to deploy it.

Bash

Run command pattern — the script exists on the local filesystem:

\[source,bash]
\----
bash $(find data/d001/projects/<slug>/scripts -name 'script-name.sh')
\----

The $(find) command resolves the absolute path portably. No $ prompt glyph.

Alias resolution — shell aliases do not exist in non-interactive bash:

Alias Resolution Variable

build-adoc

BUILD_ADOC="${BUILD_ADOC:-$HOME/atelier/_bibliotheca/domus-asciidoc-build/bin/build-adoc.sh}"

d001

Shell function — scripts that need d001 MUST document the dependency and fail with a clear message

PowerShell — Deployer Pattern

PowerShell scripts target remote Windows machines that do not have the repository. The deployer pattern uses a heredoc wrapper to transfer the script content:

Three layers:

  1. The script — scripts/<name>.ps1 — pure module code. No wrapper. Functions, query libraries, connect logic. Independently dot-sourceable: . .\<name>.ps1.

  2. The deployer — a Set-Content heredoc that wraps the script for one-paste deployment to a target machine.

  3. The partial — partials/<section>/<name>.adoc — shows the deployer block with include:: pulling the script into the heredoc at build time.

Script file (scripts/dcr-inventory.ps1):

# Pure module — no wrapper, no Set-Content
# Independently dot-sourceable: . .\dcr-inventory.ps1

$Queries = @{
    SyslogSources = @"
Syslog | summarize Count=count() by Computer | order by Count desc
"@
}

function Invoke-DCRQuery {
    param([Parameter(Mandatory)][string]$Query)
    # ...
}

function Get-SyslogSources {
    Invoke-DCRQuery $Queries.SyslogSources
}

Partial (partials/operations/dcr-inventory.adoc):

==== DCR Inventory Module

\[source,powershell]
\----
$script = @'
Unresolved include directive in modules/ROOT/partials/standards/workflow/script-extraction.adoc - include::{scriptsdir}/dcr-inventory.ps1[]
'@
$script | Set-Content "$HOME\Downloads\pwsh\dcr-inventory.ps1"
\----

At build time, include:: injects the script content into the heredoc. The reader copies the entire block, pastes into PowerShell, and the script lands on the target machine ready to dot-source.

Why this works:

Concern Mechanism

Portability

Set-Content + here-string (@'…​'@) avoids quoting issues — single-quoted here-string passes content verbatim, no variable expansion.

Testability

The .ps1 file is independently testable: . .\dcr-inventory.ps1 then call Get-SyslogSources.

Single source

The script exists once in scripts/. The partial references it via include::. No duplication.

Self-documenting

The module prints a function inventory banner on load — the user sees what’s available without reading documentation.

PowerShell Requirements

  1. PowerShell scripts in scripts/ MUST be pure module code — no Set-Content wrapper, no $script = @' heredoc.

  2. The deployer wrapper (@'…​'@ | Set-Content) MUST live in the AsciiDoc partial only, using include::{scriptsdir}/<name>.ps1[] to pull the script content.

  3. Every .ps1 module SHOULD print a function inventory banner on load (the Write-Host block listing available functions).

  4. Query libraries MUST use a $Queries hashtable with named KQL here-strings — functions delegate to a single Invoke-*Query wrapper.

  5. All functions MUST use -ErrorAction SilentlyContinue with explicit null checks — no unhandled exceptions.

  6. Read-only inventory modules MUST NOT contain New-Az*, Set-Az*, Update-Az*, or Remove-Az* cmdlets. Operational scripts that modify resources are separate files with explicit -DryRun support.

Cross-Platform Summary

Language Script Location Run/Deploy Pattern Partial Shows

Bash

scripts/<name>.sh

bash $(find …​ -name '<name>.sh')

Run command only

PowerShell

scripts/<name>.ps1

@'…​include…​'@ | Set-Content

Deployer heredoc with include::

Python

scripts/<name>.py

python3 $(find …​ -name '<name>.py')

Run command only (same as bash)

Encryption

Scripts in data/d001/ projects contain sensitive values (subscription IDs, tenant IDs, workspace names). The d001 open/close workflow only handles partials/*.adoc files. PowerShell and bash scripts in scripts/ require manual encryption:

# Encrypt all ps1 scripts
for f in data/d001/projects/<slug>/scripts/*.ps1; do
  encrypt-file "$f"
done

# Encrypt all sh scripts
for f in data/d001/projects/<slug>/scripts/*.sh; do
  encrypt-file "$f"
done

Always rm the stale .age file before re-encrypting. Old .age silently overrides edits. See the d001 workflow documentation for the canonical encrypt/decrypt cycle.

Related