Deploy on the High Side

The last mile, unattended

The package crossed (5.1), the transfer verified it (5.2); now something has to run it — on agents inside the enclave, operated by people who didn’t build it, against infrastructure you can’t see. The reference implementation is a parameterized deployment pipeline for exactly this seat, and its shape is a study in designing for the operator you’ll never meet:

THE HIGH-SIDE DEPLOYER:
  1. skip source checkout           there is no source here - the
                                    package is the truth
  2. fetch NEWEST package           from the landing container, by
                                    convention: latest-by-name
  3. extract + AUTO-DISCOVER        find the IaC in the archive: which
                                    tool (template-vs-config priority
                                    rules), which parameter files
  4. TYPE-GATED VALIDATION          per discovered tool, the dry-run
                                    that tool offers (what-if / plan) -
                                    results surfaced BEFORE apply
  5. apply                          then summarize; on failure - log
                                    (and there is the gap)

Two design choices deserve their moment. Auto-discovery makes the deployer generic: it finds .bicep or .tf, detects parameter files, chooses validation per type — so one pipeline serves every package that follows the convention, and the convention (not documentation) is the interface. Type-gated validation honors an invariant this training has hammered: never apply what you haven’t dry-run in the target environment — the low-side rehearsal (5.1) proved the package internally coherent; only the high-side plan proves it coherent here.

State travels in the package

The quiet radical decision: IaC state rides inside the archive. No state backend spans the gap (that’s the point of a gap), so the first package seeds the state file, and from then on state lives on the high side, updated in place by each apply — subsequent archives carry configuration, never a replacement state file (the low side can never know post-apply state — resource IDs and computed attributes don’t come back across a one-way gap — so a package that tried to carry authoritative state would be stale by the second cycle, and its apply would try to drag the enclave backward). Protect that high-side state like the secret-bearing artifact it is: locked, backed up before every apply, never exported. This works — the reference system runs it — under one discipline that is really Lesson 5.2 wearing infrastructure clothes: the state is only truthful if every change flows through the one-way pipeline. Any out-of-band high-side change makes the packaged state a liar, and IaC tools act on what state tells them — the next apply will try to “correct” reality back toward stale state, which in an enclave can mean deleting something an operator built for a reason. Drift detection (a plan run that should show no changes when nothing was transferred) is therefore not hygiene here; it’s the alarm that the one-way property broke somewhere upstream.

The gap: failure only logs

The reference deployer’s honest wart, called out by its own operators: on failure, it logs — there is no rollback. A half-applied infrastructure change in an enclave, with the build team on the other side of a guarded gate and the next transfer window days away, is the worst room in this training. Closing the gap is your exercise, and the shape of the close:

BEFORE apply:  capture current state + a reverse-plan artifact
               (what would undo this change) while the information
               to compute it still exists
DURING:        apply in dependency-ordered stages with a checkpoint
               after each - failure names the completed stages
AFTER failure: the operator gets a DECISION, not a log: retry the
               failed stage / execute the captured reverse path /
               hold - each as a runnable command, with REVERSIBILITY
               marked per stage (deletions, data migrations, and
               credential rotations may have no reverse path - say so
               BEFORE the stage runs, not after), in the runbook
               the package carried (5.1's rule: the runbook rides
               inside the tooling)

The deeper point generalizes past enclaves: rollback is a plan you capture before you need it, not a feature you discover you lack. The moment to compute “how do I undo this” is while the pre-change state still exists — after failure, half the information is gone. (The deploy-decoupled-from-release discipline — flags, progressive rollout, five-minute reversibility — is the DevSecOps Foundations training’s lane; here the constraint is harder: no traffic-shifting between sides of an air gap. The reverse path must be in the package.)

Prompt first: close it yourself

Here is our high-side deployer pipeline (YAML + scripts). Harden the
failure path:
1) add pre-apply capture: current state snapshot + generated reverse
   plan, stored beside the package's evidence folder, BEFORE any
   apply touches the environment;
2) stage the apply by dependency order with named checkpoints; on
   failure, emit the operator decision block - retry-stage / reverse /
   hold - as concrete commands with their preconditions stated;
3) add the drift alarm: a scheduled plan run that pages when it shows
   changes with NO transfer since the last run - one-way flow broke,
   find where;
4) rehearse the reverse path on the low side (5.1 discipline): break
   a staged apply deliberately in the rehearsal environment and prove
   the captured reverse plan actually restores - evidence into the
   package.

Step 4 is the one that separates a rollback plan from a rollback hope: a reverse path that has never executed is documentation, not capability.

Key takeaway

The high-side deployer is convention-driven and unattended: newest package, auto-discovered IaC, type-gated dry-run, then apply — with state traveling in the archive, truthful only while every change rides the one-way flow (drift with no transfer is your alarm, not noise). Close the reference design’s honest gap before it closes on you: capture the reverse path pre-apply, stage with checkpoints, hand operators decisions instead of logs, and rehearse the rollback until it is capability rather than hope. Module 5 complete — the software is across, running, and recoverable. Next: the loop back toward continuous.