Reproducible or It Did Not Happen

The question that arrives eight months later

“In March you reported 47 circuits above threshold. Where did that come from?”

It arrives from a regulator, an auditor, a lawyer, or a new executive re-examining a decision. You will not remember March. The table now holds a recomputed value — maybe 44, because a source backfilled and your logic changed twice.

Being able to answer is reproducibility, and it is a property you build in or do not have. It cannot be retrofitted after the question.

Prompt first: instrument the run

Add run recording to my transform.

Capture per run: run id, period, the git commit of the transform,
all parameter values, a source manifest (row counts and max
ingested_at per source table), output row count, assertion results,
reconciliation result including any explained variance, and publish
timestamp.

Then write me the query that answers: "we published 47 for March -
show me the logic, inputs, and parameters that produced it."
If any part of that question cannot be answered from what you have
captured, tell me what is missing rather than approximating it.

That final clause matters. The gaps are the finding. An agent that quietly approximates a missing manifest with a current-state query produces a record that looks complete and answers the eight-month question wrongly — which is worse than having no record, because you will trust it.

Stop and escalate if the raw zone turns out not to be immutable, or the time-travel retention is shorter than your audit horizon — both are platform-team facts to fix or confirm in writing, because every reproducibility claim you make rests on them.

Three things, all required

1. The logic as it was. Your transform in version control, with the commit that produced March’s run identifiable. A transform living only in a chat history or a notebook someone edits in place fails here immediately.

2. The inputs as they were. Harder. Sources change under you: backfills, corrections, late arrivals. Reproducing March means knowing what the source held when March ran, not what it holds now.

3. The parameters as they were. The threshold was 3.5 in March and is 4.0 now, because the owner changed it in June. Without that, you can run March’s logic over March’s data and still get a different answer.

Miss any one and you can produce a number, not the number.

Making inputs reproducible

Three approaches, in rough order of cost:

Rely on the raw zone’s immutability. If raw is genuinely append-only and carries an ingestion timestamp, you can reconstruct the input as of any date: WHERE ingested_at <= '2026-04-01'. This is the cheapest option and it works — provided nobody has ever done a corrective overwrite in raw. Verify that before depending on it.

Snapshot what you consumed. Keep the staging input for each run, or a manifest of source row counts and checksums per partition. Storage is cheap; a manifest is very cheap and often enough — it lets you prove the input differed even when you cannot fully rebuild it.

Table-format time travel. Some formats let you query a table as of a past version or timestamp. Convenient, and check the retention window before trusting it — the default is often shorter than your audit horizon, and a version older than retention is simply gone.

The run record

One row per run, written by the job itself:

RUN RECORD - circuit_monthly

run_id            2026-04-01T05:12:03Z-a7f2
period            2026-03
transform_commit  4b8e21c
parameters        {"threshold": 3.5, "lookback_days": 7}
source_manifest   circuit_faults: 284,109 rows through 2026-03-31,
                  max ingested_at 2026-04-01T04:58Z
                  circuit_master: 1,847 rows, checksum 9e21...
output_rows       1,844
assertions        all passed (freshness, volume, grain, referential, range)
reconciliation    OMS 3,731 vs ours 3,847, +3.1% - EXPLAINED:
                  decommissioned circuits included per definition
published_at      2026-04-01T05:14:41Z
published_by      circuit_monthly_job

That block answers the eight-month-later question completely: the logic (4b8e21c), the inputs (manifest), the parameters (3.5), and the fact that a known variance was understood at the time rather than discovered later. It costs a few lines in the job and it is the difference between an answer and an apology.

The stranger test

The standard worth holding to: could someone who has never met you follow this to a verdict?

Not “could you reconstruct it” — you have context nobody else has. A stranger needs:

• the published figure and its period
• the run record above
• the transform at that commit
• the definition as it read at that time
• the acceptance test and whether it passed

If those five exist and connect, the answer is yes. If any requires asking you, reproducibility depends on your continued employment, which is not a property of the data product.

Key takeaway

Reproducing a published number requires three things, all of them: the logic as it was, the inputs as they were, and the parameters as they were. Versioned SQL alone gets you a number computed rigorously from the wrong inputs. Record each run with its commit, parameters, source manifest, assertion results, and any explained variance — a few lines in the job that answer the question an auditor asks eight months later. Hold to the stranger test: if any part of the chain requires asking you, the property belongs to you rather than to the data product. Module 5 puts the result in front of the person who asked for it.