The Retrieval Contract

The contract, in three lines

Everything a retrieval system must do, stated without naming any technology:

IN   a question, and who is asking
OUT  passages from the corpus that may bear on it - each carrying
     openable provenance (document, revision, page), each one the
     asker is permitted to see
AND  nothing else reaches the answering model

That is the whole contract. Every implementation choice in this module is a way of satisfying it; every evaluation in the next module is a way of checking it; and it will still be the contract when every tool named in the next lesson has been replaced. The third line is as binding as the first two — the answering model sees retrieved, authorized passages, not the corpus, not the internet, not its memories of training data presented as your documents.

The shapes, as a decision table

How the OUT is produced is a genuine decision in 2026 — not because the field is confused, but because corpora and question classes differ. Four shapes, each honest about what it costs:

Shape	How it works	Right when	Wrong when
Whole-corpus context	the documents fit in the model’s working context; skip retrieval entirely, supply everything	corpus slice is small (a handful of documents per class)	corpus grows past the context, cost per question scales with corpus size
Lexical search	keyword/term matching with ranking — the forty-year-old technology, still excellent	queries share vocabulary with documents (standards, manuals, part numbers)	askers phrase questions in words the documents never use
Semantic search	passages and questions mapped into a similarity space; matches meaning, not words	vocabulary gap is real (“burnt contacts” should match “arcing damage”)	you cannot explain WHY a passage matched; adds an index to build, version, and pay for
Hybrid	lexical and semantic candidates merged	the classes genuinely mix both needs — common in practice	complexity budget is spent on retrieval instead of evaluation

Two rules govern the choice:

Start with the simplest shape the class survives. The failure-history class at Meridian has a vocabulary gap (field notes are idiosyncratic), so it earns semantic search. The standards class does not — part numbers and clause references are exact-match territory where lexical search is not the compromise but the correct tool. And a class served by six documents earns whole-corpus context: no index, no retrieval failure mode, nothing to version.

The shape is per class and recorded in the manifest. Like every Module 2 decision, it is revisitable — a corpus that grows can graduate from whole-context to indexed retrieval, and that migration is a corpus-version increment with a re-eval, not a quiet swap.

⚠The default is the failure

The industry default is to reach for the most sophisticated shape — semantic everything — because it is what the tutorials build. Sophistication is not the risk; unexamined sophistication is. A semantic index you cannot explain, on a class that needed exact clause matching, produces retrieval failures nobody can diagnose — Module 4’s “which half failed” becomes unanswerable when nobody can say why a passage was or was not returned. Choose the shape you can explain to the person who will debug it.

Prompt first, for the decision

Here are my question classes, their corpus slices, and 5 example
questions each [paste].

For each class, recommend a retrieval shape from: whole-corpus
context, lexical, semantic, hybrid. Justify by:
  - corpus slice size vs a working context
  - vocabulary gap: do my example questions use the documents' own
    words? Show the specific mismatches if not.
  - explainability: who debugs a miss, and what would they need to
    see?

Recommend the SIMPLEST shape each class survives. If you recommend
anything beyond lexical, state the specific question from my
examples that lexical would fail, and why.

That last requirement — name the failing question — is what keeps the recommendation honest. “Semantic is more capable” is always true and never a reason; “question 3’s ‘burnt contacts’ appears nowhere in the field notes, which say ‘arcing damage’” is a reason.

Stop and escalate when the shape decision implies infrastructure someone must run and pay for — a semantic index is a service with a lifecycle, and adopting one is a platform-team conversation, not a builder’s default. The class can often ship on a simpler shape while that conversation happens.

KNOWLEDGE CHECK

The standards class - part numbers, clause references, exact terminology - is being designed. The builder proposes semantic search because 'it handles everything lexical does, plus meaning.' What is wrong with the reasoning?

Key takeaway

The contract is three lines — question and asker in; authorized passages with openable provenance out; nothing else reaches the model — and it outlives every tool. The shape that satisfies it is a per-class decision recorded in the manifest: whole-corpus context when the slice fits, lexical when the vocabulary is exact, semantic only where a named question demonstrates the gap, hybrid when the complexity is genuinely bought. Simplest shape the class survives, chosen so the person debugging a miss can explain it. Lesson 3.2 builds one — and is the only lesson in this training allowed to name the tools it uses.

What you'll learn

The contract, in three lines

The shapes, as a decision table

Prompt first, for the decision

Key takeaway