Grounded Answers From Documents Module 3 · Retrieve the Evidence

The Retrieval Contract

Updated

Intermediate

What you'll learn

~18 min
  • State the retrieval contract independently of any implementation
  • Choose a retrieval shape as a decision with tradeoffs, including the do-nothing shape
  • Recognize when the simple shape is the correct one, not the compromise

The contract, in three lines

Everything a retrieval system must do, stated as a three-line contract structure without naming any technology:

IN a question, and who is asking
OUT passages from the corpus that may bear on it - each carrying
openable provenance (document, revision, page), each one the
asker is permitted to see
AND nothing else reaches the answering model

That is the whole contract. Every implementation choice in this module is a way of satisfying it; every evaluation in the next module is a way of checking it; and it will still be the contract when every tool named in the next lesson has been replaced. The third line is as binding as the first two — the answering model sees retrieved, authorized passages, not the corpus, not the internet, not its memories of training data presented as your documents.

The shapes, as a decision table

How the OUT is produced is a genuine decision in 2026 — not because the field is confused, but because corpora and question classes differ. Four shapes, each honest about what it costs:

ShapeHow it worksRight whenWrong when
Whole-corpus contextthe documents fit in the model’s working context; skip retrieval entirely, supply everythingcorpus slice is small (a handful of documents per class)corpus grows past the context, cost per question scales with corpus size
Lexical searchkeyword/term matching with ranking — the forty-year-old technology, still excellentqueries share vocabulary with documents (standards, manuals, part numbers)askers phrase questions in words the documents never use
Semantic searchpassages and questions mapped into a similarity space; matches meaning, not wordsvocabulary gap is real (“burnt contacts” should match “arcing damage”)you cannot explain WHY a passage matched; adds an index to build, version, and pay for
Hybridlexical and semantic candidates mergedthe classes genuinely mix both needs — common in practicecomplexity budget is spent on retrieval instead of evaluation

Two rules govern the choice:

Start with the simplest shape the class survives. The failure-history class at Meridian has a vocabulary gap (field notes are idiosyncratic), so it earns semantic search. The standards class does not — part numbers and clause references are exact-match territory where lexical search is not the compromise but the correct tool. And a class served by six documents earns whole-corpus context: no index, no retrieval failure mode, nothing to version.

The shape is per class and recorded in the manifest. Like every Module 2 decision, it is revisitable — a corpus that grows can graduate from whole-context to indexed retrieval, and that migration is a corpus-version increment with a re-eval, not a quiet swap.

⚠The default is the failure

The industry default is to reach for the most sophisticated shape — semantic everything — because it is what the tutorials build. Sophistication is not the risk; unexamined sophistication is. A semantic index you cannot explain, on a class that needed exact clause matching, produces retrieval failures nobody can diagnose — Module 4’s “which half failed” becomes unanswerable when nobody can say why a passage was or was not returned. Choose the shape you can explain to the person who will debug it.

Prompt first, for the decision

Switch your workspace agent into plan mode and start with (if you do not have live exports or files handy, ask it to generate a small Meridian practice sample from the lesson’s scenario first):

Here are our question classes, their corpus slices, and example questions for each: [paste your classes and examples]. Recommend the simplest retrieval shape each class survives, and ask clarifying questions if any slice is unclear.

Check that the plan or draft covers:

  • Shape options: choosing among whole-corpus context, lexical, semantic, or hybrid per question class.
  • Size and vocabulary justification: how the corpus slice compares to working context size, and whether the example questions use the documents’ own terminology or show concrete vocabulary mismatches.
  • Explainability for debugging: who diagnoses a retrieval miss and what evidence they need to inspect.
  • Simplest-shape rule: any recommendation beyond lexical search justified by naming the specific example question that lexical matching would fail and explaining why.

If the agent recommends semantic or hybrid search by default without pointing to a concrete vocabulary mismatch, follow up with: "Default to whole-corpus context or lexical search unless you can name the specific example question whose words never appear in the target documents."

That last requirement — name the failing question — is what keeps the recommendation honest. “Semantic is more capable” is always true and never a reason; “question 3’s ‘burnt contacts’ appears nowhere in the field notes, which say ‘arcing damage’” is a reason.

Stop and escalate when the shape decision implies infrastructure someone must run and pay for — a semantic index is a service with a lifecycle, and adopting one is a platform-team conversation, not a builder’s default. The class can often ship on a simpler shape while that conversation happens.

KNOWLEDGE CHECK

The standards class - part numbers, clause references, exact terminology - is being designed. The builder proposes semantic search because 'it handles everything lexical does, plus meaning.' What is wrong with the reasoning?

Key takeaway

The contract is three lines — question and asker in; authorized passages with openable provenance out; nothing else reaches the model — and it outlives every tool. The shape that satisfies it is a per-class decision recorded in the manifest: whole-corpus context when the slice fits, lexical when the vocabulary is exact, semantic only where a named question demonstrates the gap, hybrid when the complexity is genuinely bought. Simplest shape the class survives, chosen so the person debugging a miss can explain it. Lesson 3.2 builds one — and is the only lesson in this training allowed to name the tools it uses.

Search lessons