Grounded Answers From Documents Module 5 · Authorize the Answer

The Record Is a Corpus Too

Last reviewed

Advanced

What you'll learn

~18 min
  • Design the question-and-answer record with purpose and retention, not as a default log
  • Recognize the record as a sensitive dataset that needs its own tier and owner
  • Treat answers leaving the boundary as governed egress at the answer's own sensitivity
ℹLeadership brief

What it is: deliberate design of the record the QA service keeps — what is logged, why, who reads it, when it is destroyed — and a rule for answers that leave the boundary.

What it buys: the service’s own exhaust does not become the next incident. A full log of who asked what is a map of the organization’s concerns — investigations, disputes, doubts — and it accumulates silently unless someone designs it on purpose.

What to fund: a retention decision and an owner for the record. Both are cheap now and neither can be retrofitted after the log has years in it.

Prompt first: design the record deliberately

Design the serving record for the QA service. Every field must
answer to a PURPOSE from this list, or be dropped:
P1 which-half-failed diagnosis of a disputed answer (4.3)
P2 re-evaluation when a dependency changes (Module 6)
P3 serving-mode enforcement evidence (5.1)
Proposed fields - justify or drop each: question text, asker
identity, asker tier, timestamp, question class, retrieved passage
ids, answer text, citations, serving mode, checker identity (draft
mode), corpus version, config version.
Then: a TIER for the record itself (who may read the log), a
RETENTION period per field with the reason, and the record's named
owner. Flag every field where minimization conflicts with a purpose
- do not resolve silently.

The exhaust is a dataset

Run the service for a year and you own a new document collection: every question anyone asked, when, and what they were told. Read that as an asset and it is Module 6’s diagnostic gold. Read it as an exposure and it is this:

  • The sequence of questions from the contracts manager maps a dispute in progress.
  • The failure-history questions before a rate filing show what the organization worried about, and when.
  • One curious analyst’s question log is a profile of their work nobody consented to keep.

The record is not like a sensitive dataset — it is one, assembled by the service, unclassified by anyone, growing daily. The controls this training has spent five modules building for documents apply to it in full: a tier (who may read the log — narrower than who may use the service), an owner, and a place in the manifest discipline. If you took Building Trustworthy Data Products, this is its lesson that a spreadsheet quietly becomes load-bearing — except this one becomes load-bearing and radioactive.

Minimization is a purpose test, not an instinct

The prompt’s rule — every field answers to a purpose or is dropped — is where the design happens. Applying it honestly produces non-obvious calls in both directions:

Kept, because a purpose demands it: retrieved passage ids (P1 is impossible without knowing what the model saw — 4.3’s diagnostic reads exactly this field); corpus and config versions (P2 runs on them); asker tier and serving mode (P3’s evidence that the matrix was enforced).

Challenged, because the purpose is thinner than it looks: asker identity. P3 needs it only for draft-mode checker attribution; P1 and P2 never do. A defensible design keeps identity for draft-served answers and records only tier-plus-role for direct ones — the diagnosis loses nothing, and the concern-map exposure drops by most of its weight. That trade is exactly the kind the prompt’s final line surfaces instead of burying.

Retention completes it: diagnostic fields live as long as answers might be disputed (align with the class’s decision horizon); the question-text corpus does not need to outlive its purposes just because storage is cheap. An unbounded log is a subpoena surface and a breach payload, wearing the costume of thoroughness.

Answers leave — that is egress

An answer with citations gets pasted into a board deck, forwarded to a vendor, attached to a filing. The moment it crosses the boundary, the previous training’s egress rule applies unchanged: what governs is the classification of the artifact leaving, at its own grain — which for answers means the 5.1 aggregate sensitivity, not the tiers of the cited parts.

The serving surface can make the honest path easy: an “export answer” action that records what left, to where, released by whom — the same release-record fields the site’s egress discipline always demands. What it cannot do is prevent copy-paste; what it must not do is pretend copy-paste does not happen. The record of what was served to whom is the floor that makes the eventual “how did this quote reach a vendor” question answerable at all.

⚠The default log is the worst log

Skip this lesson’s design and a log still exists — whatever the serving framework writes: probably everything, forever, world-readable to the ops team, owned by nobody. Every property of it is an accident. The failure is not logging too little; it is that the maximal log happens by default, and its exposure is discovered by the first person who goes looking with bad intent or a subpoena.

Stop and escalate when someone requests the question log for a purpose outside its charter — performance review, an investigation, “just curiosity about usage”: the record’s owner rules on it against the stated purposes, and an off-purpose read of the log is itself an access event the record must record.

KNOWLEDGE CHECK

The service logs every question with full asker identity, indefinitely, 'because storage is cheap and more data is better for debugging.' What is the strongest objection?

Key takeaway

A year of service produces a dataset nobody designed: every question, every asker, every answer — a map of the organization’s concerns that is sensitive by nature and unclassified by default. Design it with the purpose test — every field justifies itself against diagnosis, re-evaluation, or enforcement, or is dropped — keep identity only where a purpose demands it, and bound retention because an eternal log is exposure wearing thoroughness. The record gets a tier narrower than the service, and an owner. Answers that leave are egress at the answer’s own sensitivity, recorded on the way out. Lesson 5.3 prices the whole thing.

LEADERSHIP DECISION give the record an owner, a tier narrower than
the service, and a retention end - before it has
years in it
PRACTITIONER ACTION pass every field through the purpose test; keep
identity only where draft-mode attribution needs
it; record what leaves the boundary
SUCCESS MEASURE zero fields without a purpose; zero off-purpose
log reads unrecorded; retention actually deletes
Search lessons