What It Costs to Be Right

Why this lesson has no numbers in it

Cloud pricing changes, engines change, and any figure printed here would be wrong before you read it. More usefully: a published cost tells you nothing about your environment, where the variables that matter are your data volume, your query patterns, and your platform’s pricing agreement.

So this lesson teaches the measurement rather than the number. The output is a figure you produced, which is the only kind you can defend.

Prompt first: measure, never estimate

Help me measure the annual cost of one data product in MY environment.

Tell me which platform-native views or logs expose: per-job compute
for a scheduled load, per-query compute attributed to a consumer,
storage by partition including any snapshot retention, and query
history I can attribute by user.

Write the queries. Do NOT estimate any cost, do not use published
per-unit pricing, and do not fill in a number I have not measured -
if something cannot be measured from what I have, say so and tell
me where the figure would have to come from.

The prohibition is the point. Asked what a pipeline costs, an agent will produce a confident annual figure from remembered pricing. It will be plausible, specific, and unrelated to your contract — and once it is in a slide, nobody re-derives it. This is the same rule as Lesson 1.4’s expected values: the agent supplies the method, you supply every fact.

The four costs

1. Compute at build. What the load consumes each run — scanned bytes, slot-seconds, or warehouse-seconds depending on your engine. Multiply by runs per year. Every platform exposes this per query or per job; find where before you need it.

2. Compute at read. What consumers spend querying your output. Easy to overlook because it lands on someone else’s budget line, and it is frequently larger than the build cost. A dashboard refreshing on every page load for forty people can outspend the pipeline that fills it by an order of magnitude.

3. Storage, tiered over time. The table plus every historical partition plus any snapshots or time-travel retention from Lesson 4.4. Storage is cheap per unit and unbounded in duration, which is a combination that quietly accumulates.

4. Human time. The hours spent maintaining, investigating variances, and answering questions about it. Usually the largest single cost and the one nobody records. A product generating one investigation a month costs more in salary than most pipelines cost in compute.

The measurement

Do this once per product, on real runs, and record the result next to the definition:

COST - circuit_monthly, measured 2026-08 over 4 runs

BUILD      scanned ~X per run (from the platform's per-job stats),
           52 runs/year
READ       queries against this table over 30 days, from the
           warehouse query log, attributed by consumer
STORAGE    current partitions + retained history + snapshot
           retention, at the current tier price
REBUILD    one full historical reprocess, measured by running it
           once against a scratch target
HUMAN      hours logged against this product in the last quarter

The REBUILD line is worth the trouble. Run it once, deliberately, and record what it cost. That single measurement converts “we should fix the 2021 gap” from a debate into an arithmetic problem.

Cost against value

A cost figure is only useful next to the value statement you wrote in Lesson 1.2 — the decision this product changes.

Meridian’s circuit ranking feeds proactive maintenance. The comparison is its annual cost against a prevented outage, and the supervisor can tell you what one outage costs in crew overtime and regulatory exposure. If the product costs a fraction of one prevented outage, the conversation is over. If it costs more than several, something is wrong with the product or with the claim.

That second case is worth taking seriously rather than defending. A product costing more than the decision it improves is a candidate for the retirement conditions from Lesson 1.4 — and finding that out yourself is much better than having it found for you during a budget review.

Stop and escalate when a cost component cannot be measured because the logs are not exposed to you — that is a platform-team request, and the discipline is to leave the line blank rather than fill it with an estimate.

Key takeaway

Measure cost in your own environment rather than trusting any published figure, including one an agent supplies confidently from memory. Four components: build compute, read compute, tiered storage, and human time — with read cost, the rebuild, and human hours being the three routinely missed. Measure the full historical rebuild once, deliberately, because that number decides whether a known defect gets fixed or lived with. Then put the cost next to the decision the product improves, and treat “costs more than the decision is worth” as a real finding rather than something to argue away. Lesson 6.3 covers what happens when you are no longer the one maintaining it.