Modern DevSecOps Foundations Module 5 · Artifacts and Evidence

Hash It, Verify It, Promote It

Updated

Intermediate

What you'll learn

~18 min
  • Promote artifacts by content hash - build once, verify at every hop, never rebuild
  • Make promotion dependency-aware with a manifest that knows what changed
  • Stage in strict layer order so nothing arrives before what it needs

The rule this module builds on

Module 2 built the pipeline; here’s the rule about what flows through it: build once, promote the artifact, verify by hash at every hop. The anti-pattern it retires is rebuild-per-environment — where “tested in staging” describes a different binary than what production got, and nobody can prove otherwise. The reference estate treats this as a house standard across two unrelated systems (a data-platform promoter and a modernized app pipeline — when independent teams converge on a pattern, teach the convergence): generate the hash at build, record it somewhere the store cannot silently rewrite (the pipeline run record and the PR - or signed, Lesson 5.2), and re-verify before every deploy — staging’s gate, prod’s gate, each hop confirming bytes match the manifest before anything runs.

That habit sounds paranoid until you remember Lesson 3.3’s worms and moved tags: the hash check is the supply-chain boundary drawn around your own artifacts. It costs one pipeline step (a tier-1 template, naturally) and converts “we deployed what we tested” from an assumption into an assertion that fails loudly when false.

Dependency-aware promotion: the manifest pattern

For a single service, hash-and-verify is the whole story. The interesting engineering appears when a set of interdependent artifacts promotes together — the reference case is a data platform (pipelines, datasets, linked services, notebooks, triggers, all referencing each other), and its manifest pattern generalizes to any artifact family. Here is a reference JSON snippet showing the three-pool promotion manifest shape:

{
"artifacts": [ // what the human changed
{ "name": "billing-pipeline", "type": "pipeline",
"hash": "sha256:9f2c...", "dependencies": ["billing-dataset", "sql-ls"] }
],
"unchangedArtifacts": [ ... ], // the resolution pool - hashes prove
// "unchanged," nothing gets re-shipped
// just for standing nearby
"foundationalArtifacts": [ ... ] // always-present infrastructure
}

The promoter’s algorithm structure, and its three teachable properties:

1. take the CHANGED set (hash-diff decides membership, not human memory)
2. recursively WALK dependencies into the unchanged/foundational pools -
a changed pipeline pulls its dataset and linked service along ONLY if
the target lacks the right version
3. stage in STRICT LAYER ORDER: connection DEFINITIONS (secret
references, never secret values - each environment resolves its
own) -> runtime ->
linked services -> datasets -> notebooks -> pipelines -> triggers
(nothing arrives before what it references; triggers arrive LAST so
nothing fires half-assembled)
4. missing referenced artifact anywhere -> FAIL FAST, promote nothing

Property one: the hash is the change-detector — promotion scope is computed, not remembered, so the Friday deploy can’t forget the linked service someone edited Tuesday. Property two: unchanged-and-undepended artifacts simply don’t move — minimal blast radius per promotion, and the diff between environments stays meaningful. Property three: layer order is encoded, not tribal — the ordering that lives in a senior engineer’s head becomes a list in the promoter, and note the deliberate bookends (credentials first, triggers last — the enable-the-schedule-last instinct you may recognize from Cloud Modernization’s migration cutover). Property four is this training’s oldest friend: fail fast and loud beats promote-and-pray.

Switch your workspace agent to Plan mode on Meridian’s reporting stack and ask it to plan the hash-verified promotion manifest and promoter:

Plan the promotion manifest generator, dependency-aware promoter, and per-hop hash-verification step for Meridian's reporting stack. Include dry-run PR output and tests proving tamper refusal, missing-dependency fail-fast, and target drift detection.

Check that the plan or draft covers:

  • A manifest generator that hashes every artifact at build time, records declared dependencies, and splits changed from unchanged artifacts against the target environment
  • A promoter that walks dependencies recursively, stages artifacts in strict layer order (connection definitions with secret references first, triggers last), fails fast on missing references, and offers a dry-run plan for pull requests
  • A Tier 1 verify step template wired into your Module 2 deploy stages that re-hashes artifacts against the manifest at every environment hop
  • Verification tests demonstrating all three failure modes: a tampered artifact failing the hop check, a missing dependency aborting promotion, and an out-of-band target hotfix surfaced by hash diff

If the manifest embeds raw connection strings instead of environment-resolved secret references, follow up with: Store only secret references in connection definitions so each target environment resolves its own credentials.

That third failure proof deserves its sentence: the same hash-diff that computes promotion scope is a drift detector for free, because an artifact whose target-environment hash matches neither the old nor the new manifest was changed by hand, and the promoter just found your out-of-band hotfix (Zero Trust’s drift lesson, arriving via supply-chain mechanics).

💡Where this sits in the bigger integrity story

This lesson is integrity’s FIRST half — your artifacts, your hashes, your hops. The second half (5.2) is the industry-standard machinery for the same instinct: provenance attestations and signatures that make the claims verifiable by someone who ISN’T you, plus the SBOM that inventories what’s inside. Same rule — never trust an unverified artifact — widening from your pipeline to your ecosystem.

KNOWLEDGE CHECK

A release manager proposes an optimization: 'The verify step re-hashes every artifact at every hop, but the artifacts come from OUR pipeline artifact store over TLS — nothing changes them in transit. Drop the verify steps and save 40 seconds per deploy.' What's the flaw?

Key takeaway

Build once; let hashes compute what changed; promote with dependencies walked and layers ordered (connection definitions first, triggers last); fail fast on anything missing; and re-verify at every hop so tested-equals-deployed is an assertion, not folklore — with drift detection falling out for free. Next: making those integrity claims verifiable by the rest of the world.

Search lessons