What It Costs to Ask

Prompt first: measure, never estimate

Instrument the QA service to measure its own costs - measured in MY
environment; do not estimate anything, do not use remembered
pricing, and leave blank what cannot be measured with a note on
where the figure would come from.

Per QUESTION, by class: retrieval compute, answering tokens in and
out (the model bill's unit), and - for draft mode - checker minutes
from serve-to-release timestamps in the record.
Per CHANGE: the re-extraction and re-index cost of the last corpus
version bump, and the eval-run cost that followed it (screen tokens
plus hand-check minutes).
Standing: index storage, and the record's storage.

Output: cost per answered question by class and serving mode, and
cost per corpus change - as measurement queries against the serving
record and the platform's own billing surface, not as numbers you
supply.

If you took Building Trustworthy Data Products, the discipline is its cost lesson unchanged — the method travels, the numbers never do — carried here in one sentence so this lesson can spend itself on what is genuinely different: the shape of the cost.

This cost model is inverted

A data product’s cost lived in builds and storage: it cost money when it ran on schedule, roughly the same whether three people or three hundred read the result. The QA service inverts that:

Cost scales with questions asked. Every question runs retrieval and pays for answering tokens — the corpus sitting still costs almost nothing; curiosity costs per unit. A class that gets popular gets expensive, linearly, and popularity is exactly what success looks like. The serving matrix from 5.1 is therefore also a budget instrument: who may ask is the throttle, and a class opened to all-staff has had its cost ceiling opened with it.

Change bills separately. A corpus version bump re-extracts, re-indexes, and — per Module 6’s rule — re-runs the evals. That makes the cadence of corpus change a cost driver the data-product world never had: a corpus that takes weekly document drops pays the re-index-plus-re-eval bill weekly. Batching drops into a monthly version is often the single largest cost decision available, and it is a governance choice (2.4), not an engineering one.

The largest line is human. Draft mode consumes checker minutes per answer, and the serve-to-release timestamps in the record measure them precisely. Price it honestly: if the contracts class serves forty drafts a month at ten checker-minutes each, that is the real cost of serving a class whose consequences demand a human — and it is still usually a bargain against the finding-hours the retrieval saved. When it stops being a bargain, that is not a cost overrun; that is the value question, which the next module owns.

Stop and escalate when a cost line cannot be measured because the platform’s billing surface does not expose it — token spend not attributable per question, compute not broken out by service. That is the platform team’s gap to close, and the discipline holds from the previous training: the line stays blank with a name attached; an estimated figure in a cost table is a fabrication with units.

Key takeaway

The cost model inverts the data-product one: the corpus sitting still is nearly free, and spend follows questions asked — so success is the cost driver, the serving matrix doubles as the throttle, and per-question cost by class is the monthly sentence-maker. Change bills separately, at re-index-plus-re-eval, which makes corpus-change cadence a governance-owned cost decision. The largest line is checker minutes, measured from the record’s own timestamps and usually still a bargain against the finding-hours saved — and every number is measured in your environment or left blank with a name, never estimated. Module 6 takes over the day after launch.