Bound the Question Classes
Last reviewed
IntermediateWhat you'll learn
~18 min- Convert surviving questions into named, bounded question classes
- Attach a consumer and a decision to each class, and drop classes that have neither
- State what the service does with questions outside its classes
Prompt first: from survivors to classes
These questions survived the kill gate: [paste the routed survivors]
Group them into QUESTION CLASSES - a class is a set of questionsanswerable by the same kind of evidence from the same part of thecorpus. For each class give me: - a name ("equipment failure history", "standard requirements", "contract obligations") - the evidence kind it needs (field notes / manual sections / filing clauses) - who asks it (a person or role) and what they DECIDE with the answer - 3 example questions inside the class, and 2 questions that look similar but fall OUTSIDE it, with the reason
Then flag any class where you could not fill in who-asks orwhat-they-decide - do not invent a consumer to keep a class alive.The out-of-class examples are the load-bearing part, and the rest of the lesson is about why.
A service commits to classes, not to conversation
The proposal that reaches leadership is usually “a chatbot over our documents.” That framing commits you to answering anything — and a system committed to anything cannot be evaluated, because there is no boundary around what it must get right and no license to refuse what it should not touch.
A question class is the unit this training operates on: a set of questions answerable by the same kind of evidence from the same slice of the corpus. Meridian’s survivors grouped into three:
| Class | Evidence | Who asks | They decide |
|---|---|---|---|
| Equipment failure history | field notes, work orders | planning supervisor | proactive replacement vs repair |
| Standard requirements | standards editions, manuals | field engineering | what a compliant build requires |
| Contract obligations | interconnect filings, agreements | contracts manager | what Meridian owes and is owed |
Everything downstream is decided per class: which corpus slice Module 2 governs, which evaluation questions Module 4 grades, who may ask in Module 5, and — critically — what the pass bar is, because “good enough” for a failure-history browse and for a contract-obligation answer are not the same number.
The consumer and the decision, again
If you took the previous training this is its opening discipline transposed, and it earns its keep the same way: a class with no named asker and no decision is a class you delete. Not because the questions are illegitimate, but because you cannot set a pass bar for an answer nobody acts on, and Module 4 runs on pass bars.
The transposition has one new wrinkle. Data products served one consumer per product. A QA service tempts you into “everyone can ask” — and everyone is not a consumer, it is an absence of one. Each class gets its own asker and its own decision, and a class asked by everyone for nothing fails the test regardless of how interesting its questions are.
The boundary is a feature the users will fight
State, per class, what the service does with questions outside it. The honest options:
- Refuse with a route: “This looks like a counting question — the circuit reliability table answers it” (back through the kill gate’s exits, automated as a response).
- Refuse plainly: “Outside what this service answers.” Worse experience, still honest.
- Answer anyway: the dishonest option, and the default if you decide nothing — the model will produce something fluent for any question whatsoever.
The service answers failure-history questions well, so someone asks it a rate-case question, gets a fluent answer, and forwards it. Nothing failed technically — the model happily answered. What failed is that the answer was graded by nobody, against no pass bar, from a corpus slice never governed for that use.
The boundary is not there to disappoint users. It is there because every class you answer is a class you must evaluate forever, and answering an unevaluated class is exactly the fluent-wrong-answer failure from Lesson 1.1 wearing a success story.
Stop and escalate when a stakeholder wants a class added after the design is set: the addition is legitimate — classes do grow — but it repeats the whole chain (gate, corpus slice, eval questions, pass bar, authorization), and whoever owns the service decides whether that maintenance is funded, not whoever asked the question.
A proposed class - 'general utility knowledge, asked by anyone' - has enthusiastic support. What is wrong with it?
Key takeaway
A service commits to bounded question classes — same evidence kind, same corpus slice, named asker, named decision — never to conversation. Every downstream artifact is per class: the corpus slice, the eval questions, the pass bar, the authorization. A class without an asker and a decision gets deleted, and questions outside every class get refused with a route, because the model will fluently answer anything and every unevaluated answer is Lesson 1.1’s failure in a success costume. Lesson 1.4 freezes the questions the evaluation will grade — before the build exists to flatter them.