The Acceptance Test

Why this comes before the build

You now have a consumer, a decision, and a definition. The instinct is to start building. One artifact stands between you and that, and it is the cheapest insurance in this training.

An acceptance test is a written statement, agreed before construction, of what has to be true for the output to be considered correct. Not “it looks reasonable.” Something falsifiable.

The reason to write it first is not process hygiene. It is that after you build something, your judgment about whether it is right becomes permanently compromised. You will have spent two days on it. The number will look plausible. Every one of the small decisions you made along the way will feel obviously correct in hindsight. Writing the test beforehand is how you get a verdict from a version of yourself that has nothing invested.

⚠This matters more with an agent, not less

When a build took two weeks, you had two weeks of intimacy with the data and would probably notice a wrong number. When the same build takes forty minutes of prompting, you have forty minutes of intimacy and a result that looks just as polished.

Speed is the benefit. Reduced familiarity is the cost, and an acceptance test written in advance is how you pay it.

Prompt first: scaffold the test, keep the facts

Here is my metric definition [paste the five-element definition].

Draft an acceptance test with four sections:
1) a reconciliation target that is INDEPENDENT of the pipeline I am
   about to build - if you cannot identify one from what I have given
   you, say so and tell me what to go find
2) specific expected values, in a form a supervisor who does not read
   SQL could verify in 90 seconds
3) permitted use, and one plausible misuse worth writing down
4) retirement conditions

For section 2, propose the checks - I will supply the actual known
values, because you do not have them and must not invent them.

The last sentence is load-bearing. Asked for expected values, an agent will generate plausible ones. They will look exactly like real ones and they will be fabricated, and an acceptance test built on invented expected values is worse than no test — it manufactures confidence. Ask for the structure, supply the values.

💡A pattern you will use in every module

Let the agent produce the scaffolding, and reserve for yourself anything that is a fact about the world. The agent knows query syntax, table shapes, and check patterns. It does not know that DIST-4471 had a bad Q3, and when it does not know something it will produce something shaped like knowledge anyway.

Stop and escalate if no independent reconciliation target exists anywhere — no operational system, no filed figure, no supervisor who knows one case. That absence is a finding about the organization, and it goes to the metric owner before a line of the build is written.

KNOWLEDGE CHECK

You plan to reconcile your new circuit table against a summary table that a colleague built last year from the same source lake, using similar logic. What is the problem?

What goes in it

Four parts, and the first is the one people skip.

1. A reconciliation target that you did not build

Name something that already exists, that you are not about to create, that should agree with your output. Examples at Meridian:

The outage management system’s own count of fault events for the same period
The regulatory reliability report filed last year, for the overlapping months
A single circuit that a supervisor knows well, hand-checked

The requirement is independence. Reconciling a table against another table built from the same source by the same logic proves the logic is consistent, not that it is right. If the only available target is a person’s knowledge of one circuit, that is a legitimate target — write it down as such.

2. Specific expected values

Not “counts should be reasonable.” Numbers or ranges, committed in advance:

ACCEPTANCE TEST - circuit_degradation_score, monthly

RECONCILES TO   outage management system fault counts, same period
TOLERANCE       total fault events within 2% of OMS; any circuit
                differing by more than 5 events is investigated
                individually before publication

KNOWN VALUES    circuit DIST-4471 scored high in Q3 last year after the
                Elm Street failures - it must appear in the top decile
                for those months or the calculation is wrong
                circuit DIST-1102 was decommissioned in March - it must
                NOT appear after 2026-03

SHAPE           one row per energized circuit per month; row count within
                3% of the circuit master count, and the difference is
                explainable by mid-period energizations
NO-DATA         circuits with no telemetry appear with status "no data",
                never with a score of 0

The KNOWN VALUES section is the one that catches real errors. A supervisor can look at that list and tell you immediately whether it is right, and it costs them ninety seconds. Any test that requires reading SQL to evaluate is not a test the domain expert can sign.

3. Permitted use

Write down what this output may and may not be used for, at definition time. Not because a policy requires it, but because you will not remember the constraint in six months and neither will the person who inherits it.

At Meridian: “Operational maintenance prioritization only. Not for rate filings — the population excludes customer-owned equipment, which rate cases must include.”

That one sentence prevents the specific disaster where a well-built operational metric gets quietly reused in a regulatory context whose population rules are different.

4. Retirement conditions

The question nobody asks at the start: what would make this stop being worth maintaining?

The decision it feeds stops being made
The consumer moves on and nobody replaces them
A source system it depends on is replaced

Recording this now is what prevents the outcome where an organization runs 400 data products, 250 of which nobody can safely turn off because no one remembers what they were for. Module 6 returns to this; the record starts here.

Key takeaway

Write the acceptance test before you build, because afterwards your judgment is compromised by two days of investment and a plausible-looking number — a risk that grows when an agent compresses those two days into forty minutes. It needs a reconciliation target genuinely independent of the pipeline you are about to build, expected values a domain expert can verify without reading SQL, a permitted-use statement, and retirement conditions. Let the agent build the scaffolding; supply every fact about the world yourself, because an agent asked for expected values will invent convincing ones. Module 2 goes to the data itself, and finds out whether what you were handed can support any of this.

What you'll learn