The Questions Nobody Can Answer

The question the data products cannot touch

If you took Building Trustworthy Data Products, Meridian Utilities can now prove its numbers: the circuit rankings reconcile, the loads rerun identically, every metric has an owner. This training stands alone — that one sentence is all the catching up you need.

Last month the distribution planning supervisor asked a new question:

“We replaced the regulator bank on DIST-4471 in 2019. What did the crew actually find inside it?”

No table holds that. The answer exists — in a field note someone typed in a truck seven years ago, or on page 40 of a maintenance manual explaining what that failure mode looks like, or in a filing from the interconnect contract. Meridian owns roughly forty thousand such documents: manuals, standards, field notes, regulatory filings, agreements. Nobody can query any of them.

This is the other half of the gap the last training closed. Structured data answers how many and which ones. Documents answer what happened, what does the standard require, and what did we agree to — and in most organizations those answers are effectively lost the day the document is filed.

What this training builds, stated carefully

One grounded question-answering capability over a governed collection of documents — a corpus. You ask a question; the system finds relevant passages and produces an answer that cites the passages it used, with citations a reader can open.

The word doing the work is grounded, and it is worth being precise about what it buys, because this is where most such projects go wrong.

⚠The specific way this technology fails

An AI system answering from documents produces fluent, confident prose whether or not the answer is right. A wrong answer does not look wrong. It looks exactly like a right answer, cites a real document, and reads well — and somebody acts on it.

That is worse than no answer. No answer sends the supervisor to ask a person. A fluent wrong answer ends the search.

The modest claim

Here is the entire thesis of this training, and it is deliberately smaller than what vendors claim:

A grounded answer does not make the answer right. It converts an unbounded verification problem — “is this true?” — into a bounded one: “does this cited passage say this, and is this passage current and authoritative?”

That bounded check is one you — the person who knows the domain — can perform in about ninety seconds: open the citation, read the passage, decide whether it says what the answer claims. You could not have verified the claim from nothing. You can verify it against a page.

Everything else in this training exists to keep the capability inside that claim: evaluation questions written before the build, a governed corpus, citations that open, a system that says “not in the corpus” instead of guessing, and a hand check that no automation replaces. When you cannot hold the capability inside the claim, this training will say so plainly — that is Module 4’s closing lesson, and it is the one the leadership brief points at.

What you will build

Over six modules, one capability, end to end:

Module 1 decides which questions deserve this at all — most do not — and freezes the evaluation questions before anything is built.
Module 2 governs the corpus: what is in, what is authoritative, who may see what, and the extraction step where real corpora die.
Module 3 builds the retrieval, with an AI CLI, on a practice corpus of real public utility documents.
Module 4 proves answers against their cited sources — and teaches which half failed when they do not.
Module 5 decides who may ask what, because a synthesized answer can be more sensitive than any passage it cites.
Module 6 keeps it honest: any changed dependency re-runs the evaluations, and retirement is a feature.

The practice corpus is real: public maintenance manuals, decades of electric standards with superseded editions, interconnection filings — documents with genuine tables, genuine conflicts, and genuine mess, because the defects are the teaching material.

KNOWLEDGE CHECK

A grounded QA system answers the supervisor's question fluently, citing a real maintenance manual. What has the citation established?

Stop and escalate before you go further if nobody in your organization can say who owns the documents you would build on — corpus ownership is the first question of Module 2, and if the answer is “nobody,” that is a finding for leadership, not a gap you can design around.

Key takeaway

Structured data answers how many; documents answer what happened and what did we agree to — and most organizations have lost the ability to query the second kind. A grounded QA capability makes a deliberately modest claim: not that its answers are right, but that every answer comes with a bounded, ninety-second check — does the cited passage say this? Fluent wrong answers are this technology’s specific failure, worse than no answer because they end the search. Lesson 1.2 starts where every good build starts: with the four ways to answer the question that require no build at all.