Hash It, Verify It, Promote It

The rule this module builds on

Module 2 built the pipeline; here’s the rule about what flows through it: build once, promote the artifact, verify by hash at every hop. The anti-pattern it retires is rebuild-per-environment — where “tested in staging” describes a different binary than what production got, and nobody can prove otherwise. The reference estate treats this as a house standard across two unrelated systems (a data-platform promoter and a modernized app pipeline — when independent teams converge on a pattern, teach the convergence): generate the hash at build, record it somewhere the store cannot silently rewrite (the pipeline run record and the PR - or signed, Lesson 5.2), and re-verify before every deploy — staging’s gate, prod’s gate, each hop confirming bytes match the manifest before anything runs.

That habit sounds paranoid until you remember Lesson 3.3’s worms and moved tags: the hash check is the supply-chain boundary drawn around your own artifacts. It costs one pipeline step (a tier-1 template, naturally) and converts “we deployed what we tested” from an assumption into an assertion that fails loudly when false.

Dependency-aware promotion: the manifest pattern

For a single service, hash-and-verify is the whole story. The interesting engineering appears when a set of interdependent artifacts promotes together — the reference case is a data platform (pipelines, datasets, linked services, notebooks, triggers, all referencing each other), and its manifest pattern generalizes to any artifact family:

{
  "artifacts": [                       // what the human changed
    { "name": "billing-pipeline", "type": "pipeline",
      "hash": "sha256:9f2c...", "dependencies": ["billing-dataset", "sql-ls"] }
  ],
  "unchangedArtifacts": [ ... ],       // the resolution pool - hashes prove
                                       // "unchanged," nothing gets re-shipped
                                       // just for standing nearby
  "foundationalArtifacts": [ ... ]     // always-present infrastructure
}

The promoter’s algorithm, and its three teachable properties:

1. take the CHANGED set (hash-diff decides membership, not human memory)
2. recursively WALK dependencies into the unchanged/foundational pools -
   a changed pipeline pulls its dataset and linked service along ONLY if
   the target lacks the right version
3. stage in STRICT LAYER ORDER: connection DEFINITIONS (secret
   references, never secret values - each environment resolves its
   own) -> runtime ->
   linked services -> datasets -> notebooks -> pipelines -> triggers
   (nothing arrives before what it references; triggers arrive LAST so
   nothing fires half-assembled)
4. missing referenced artifact anywhere -> FAIL FAST, promote nothing

Property one: the hash is the change-detector — promotion scope is computed, not remembered, so the Friday deploy can’t forget the linked service someone edited Tuesday. Property two: unchanged-and-undepended artifacts simply don’t move — minimal blast radius per promotion, and the diff between environments stays meaningful. Property three: layer order is encoded, not tribal — the ordering that lives in a senior engineer’s head becomes a list in the promoter, and note the deliberate bookends (credentials first, triggers last — the enable-the-schedule-last instinct you may recognize from Cloud Modernization’s migration cutover). Property four is this training’s oldest friend: fail fast and loud beats promote-and-pray.

The artifact: build the promotion manifest + promoter for Meridian's
reporting stack (six artifacts across four layers with real dependencies):
1. the manifest generator - hash every artifact at build, record declared
   dependencies, split changed/unchanged against the target environment
2. the promoter - dependency walk, layer-ordered staging, fail-fast on
   missing references, and a DRY-RUN mode that prints the promotion plan
   (what moves, what's pulled in, what stays) for the PR description
3. the verify step template - re-hash against the manifest at each hop,
   wired into the deploy stages from Module 2
4. prove all three failure modes: a tampered artifact (hash mismatch at
   verify), a forgotten dependency (fail-fast), and a manual hotfix in the
   target (hash-diff flags the drift on next promotion)

That third proof deserves its sentence: the same hash-diff that computes promotion scope is a drift detector for free — an artifact whose target-environment hash matches neither the old nor the new manifest was changed by hand, and the promoter just found your out-of-band hotfix (Zero Trust’s drift lesson, arriving via supply-chain mechanics).

💡Where this sits in the bigger integrity story

This lesson is integrity’s FIRST half — your artifacts, your hashes, your hops. The second half (5.2) is the industry-standard machinery for the same instinct: provenance attestations and signatures that make the claims verifiable by someone who ISN’T you, plus the SBOM that inventories what’s inside. Same rule — never trust an unverified artifact — widening from your pipeline to your ecosystem.

KNOWLEDGE CHECK

A release manager proposes an optimization: 'The verify step re-hashes every artifact at every hop, but the artifacts come from OUR pipeline artifact store over TLS — nothing changes them in transit. Drop the verify steps and save 40 seconds per deploy.' What's the flaw?

Key takeaway

Build once; let hashes compute what changed; promote with dependencies walked and layers ordered (connection definitions first, triggers last); fail fast on anything missing; and re-verify at every hop so tested-equals-deployed is an assertion, not folklore — with drift detection falling out for free. Next: making those integrity claims verifiable by the rest of the world.

Hash It, Verify It, Promote It

What you'll learn

The rule this module builds on

Dependency-aware promotion: the manifest pattern

Key takeaway