Prove What Is Running

Before the detail — Decision: join runtime evidence to the record version in force. Outcome: which version answered last Tuesday, answered in one query. Artifact: the join query and the sampled review note. Status of what follows: periodic human review binding for federal high-impact use; reusable guidance elsewhere.

Prompt first: draft the join, invent no data

Here is section 6 of the release-decision record for MU-AI-004
[paste] and the FIELD NAMES our trace pipeline emits for AI calls
[paste names only - no values].

Draft the JOIN SHELL that answers "which version answered request X
on date D, and was the decision in force at that moment?":
  FROM TRACES    request id; system id; timestamp; the field naming
                 the responding model or runtime digest; the deployment
  FROM REGISTER  system id; every record VERSION for it with its
                 effective interval (approved-from, superseded-or-
                 withdrawn-at) and transition history; each version's
                 pinned boundary and valid-through
  JOIN ON        system id + deployment; COMPARE the trace's
                 responding-version field to the record's boundary
  RETURN         two verdicts, separately: BOUNDARY = MATCH /
                 MISMATCH / UNJOINABLE (trace has no version field);
                 DECISION = the record version in force at the
                 timestamp and its state then, or UNRECORDED (no
                 version covers the deployment at that time)

Use only the field names I pasted. Where our traces have no field
for a column, write MISSING; do not invent one. No sample rows.

The question always arrives at the wrong time — a customer complaint, an assessor’s sample, a regulator’s letter — and the answer must come from two tables, not from memory. The agent can draft the join; only the platform team knows which trace fields exist, and a MISSING column found now is a Tuesday saved later.

Last Tuesday’s answer is a join, not a memory

Module 1 opened on the gap: nine of Meridian’s ten systems could not say which model version answered a customer on a given day. Two tables exist by now — the runtime evidence the platform emits and the approved record the register holds — and this lesson adds the join between them, with a boundary verdict and a decision verdict per request:

• MATCH — the version the trace names is inside the boundary of the record version in force at that timestamp. Boundary only: the decision verdict beside it says which lifecycle state that version was in (approved, approved-with-conditions, suspended, stale) and whether it was current.
• MISMATCH — the trace names a version the record does not pin: 6.1’s evidence-invalidating trigger, found after the fact.
• UNJOINABLE — the trace carries no responding-version field, so the boundary verdict cannot be formed; a finding about the platform, and every request under it is unproven.
• UNRECORDED — traces exist for a deployment no record version covered at that timestamp — not merely “no record today”: a request from last Tuesday is judged against the version in force last Tuesday, whatever happened since: 1.3’s fourth exit, already running and uninventoried: safe discontinuation unless a pilot exemption or waiver applies, or a remediation deadline commercially.

Where the logs live is Zero Trust 5.1 (a separate training in this series), which centralizes the logs and proves the alert rules fire. What one answer carries about its inputs is Grounded Answers 6.1, where every answer names the versions that produced it. This lesson joins their output to the register.

The field that names the version

OTel (the open standard for traces and logs) publishes conventions for generative-AI calls, among them a response-model field: the model that actually answered, as the provider reported it, separate from the model the request asked for.

Two facts belong in the record. The conventions’ stability level is Development, not stable: field names can still change, so the platform team pins the convention version it emits and expects a migration. And they are written for provider-hosted models; for MU-AI-004’s local runtime the equivalent is the digest section 2 already pins, and the join compares digests, not names.

A trace with no responding-version field cannot be joined — every request it records is UNJOINABLE by construction (a deployment that has a record but whose runtime cannot be tied to it; distinct from UNRECORDED, a deployment with no record at all), and “cannot say which version answered” becomes a finding about the platform, not just the register. The field itself has a name in the OpenTelemetry generative-AI conventions — gen_ai.response.model — and those conventions are at Development stability, so the name may still move.

Evidence, not proof

The response-model field says what the provider’s interface reported. It can report an alias. It can report a name that held while the behavior behind it moved. 2.2’s rule stands: a name proves no version — and the trace field inherits it. A MATCH on a name is evidence that the provider asserted the expected version, not proof that the same weights answered.

The canary from the changed-dependency feed compares outputs byte-for-byte on frozen probes; if the trace names the expected version and the canary drifted the same day, the trace is the weaker witness. Contract change notices, where they exist, are a second. Byte-identical output from the mock in the workbench is the third — and it proves only that the same probes produced the same bytes: corroborating evidence of an unchanged version, never proof, and only for the mock.

Treating a provider-reported name as proof converts a vendor’s assertion into Meridian’s signed claim — and the contract exposure when it turns out to be an alias is Meridian’s.

The periodic human review

M-25-21 (the White House budget office’s April 2025 memo governing agency AI use) requires ongoing monitoring, and testing and periodic human review where feasible, and the join makes the review concrete: a sample of live traces, read by a person, checked against section 4’s conditions. Did the responding version stay inside the boundary? Was the hand-check sample attached? Was the feedback channel used? Is the halt procedure’s tested date current? The cadence is set per system in section 1 — M-25-22 (the same office’s memo governing AI purchases) offers “quarterly or biannual” as examples chosen by program need — and the output is a dated register entry.

The review is a read, not a re-run. It does not move the last re-evaluation date; it can fire a trigger that does. A MISMATCH in the sample suspends under 6.1’s rule; an unmet condition suspends too, under the failed-condition action section 4 wrote for it (5.3: every condition names what happens when it fails). If the sample turns up an answer that harmed someone, the review has found an incident — its clocks, its command, and who is on call belong to the next chapter, and the review hands it there with the trace attached.

The sampled read costs the reviewer hours that 6.3’s portfolio sheet budgets per system; skipping it means discovering at renewal that a condition lapsed months ago, which reopens the whole decision.

Stop and escalate when the join returns UNRECORDED for a production deployment — traces prove something is answering real users and no record covers it. That is 1.3’s fourth exit opening — safe discontinuation, a pilot exemption or waiver, or a commercial remediation deadline — and the risk acceptor for that business area decides which.

The commercial starting practice is observability: traces with a model field, read when something breaks. The federal delta is M-25-21’s periodic human review of live outputs against the record — a scheduled read, not an incident response — and, for Meridian’s federal-task-order use case, the contract’s monitoring rights. The handoff artifact is the join query and the sampled review note; what is not equivalent is that a commercial trace exists for debugging while the federal review exists to be shown to an assessor.

Practice status — among organizations running AI systems under a signed record, commercial and federal

Scale: required | common baseline | strong optional | reference-shop (seen only at organizations that publish their own practice) | emerging

Key takeaway

“Which version answered request X last Tuesday” is a join between the traces the platform emits and the record the register holds — a boundary verdict (MATCH, MISMATCH, UNJOINABLE) and, separately, the record version in force at that timestamp or UNRECORDED; a mismatch is a trigger found late, an unrecorded deployment is the safe-discontinuation exit, an unjoinable trace is a platform finding. The OTel response-model field makes the join possible, at Development stability, so pin the convention version; for a local runtime, join on the digest. A provider-reported name is evidence, never proof of the weights behind it. The periodic human review is a sampled read against section 4’s conditions; it can fire triggers, but only a re-run moves the re-evaluation date. Lesson 6.3 steps back from one record to ten and reads the bill.

LEADERSHIP DECISION   require that every production AI deployment
                      emit a responding-version field that joins to
                      its record - a system that cannot say what
                      answered is one you cannot sign for
PRACTITIONER ACTION   build the join shell from real trace fields,
                      pin the convention version, corroborate every
                      name-match with the canary, run the sampled
                      review on the record's cadence
SUCCESS MEASURE       any request's responding version and record
                      state answered in one query - reviewer minutes,
                      not an investigation - and zero UNRECORDED or
                      UNJOINABLE production deployments at each review