The Model Evidence Packet

Before the detail — Artifact: the evidence packet, accepted when every claim has a source and every gap is marked. Status of what follows: binding for federal LLM procurement (M-26-04); reusable guidance elsewhere.

Prompt first: extract claims, omissions, applicability

Here is the vendor's model card for MU-AI-005 [paste] and the
packet template [paste].

Fill the packet STRUCTURE only:
  CLAIMS         - each statement on intended use, evaluation
                   method, attack results by attempt count, named
                   external evaluators, known limits - quoted
  OMISSIONS      - each of the five not addressed, written
                   "not addressed" - never inferred
  APPLICABILITY  - one BLANK judgment field per claim: does it
                   cover Meridian's use?
  CONTRACT FACTS - rows for no-training, retention, indemnity
                   conditions, portability, change notice - blank

A fact the card does not state is an OMISSION - write "not
addressed". A judgment (does this apply to us?) is NEEDS-OWNER.
A contract value comes only from the signed agreement - CONTRACT
FACT or "absent". Never blend the three.

A card is a persuasive document, and an agent reading one drifts toward restating its confidence as fact. The agent structures and quotes; A. Whitfield, who owns MU-AI-005, confirms the facts; the risk acceptor for that system (NEEDS-OWNER — the register names an owner, not an acceptor) judges applicability; every contract value comes from the signed agreement.

The seat you read from

Lesson 1.4 put the signature on the risk acceptor, and the packet is read from that seat. It renders no verdict on the supplier and drafts no solicitation. The buyer’s side of the table (solicitation, remedies, exit rights) belongs to a later practice area on software acquisition; this lesson records approved contract facts and stops when a required fact is absent.

Up to six documents: the model card (purpose and evaluation), the system card (the deployed service’s filters, routing, tools), the data card (training data and rights) — it is uncommon for a vendor to produce all three, and each absence is recorded — then the AUP (the vendor’s acceptable-use policy), the end-user resources, and the feedback channel. Each is read as Grounded Answers 4.4 (a separate training in this series) taught — a screen, not a verdict — so the columns are claims and omissions, never pass and fail.

Five things to extract from a card

1. Intended use — in the vendor’s words; whether Meridian’s use fits is the owner’s call.
2. Evaluation method — what was tested, on what data, by whom.
3. Attack results by attempt count — never a single number; the curve is the bar, the cards that publish one set it, and the figures live in Lesson 2.2, quoted there as vendor claims.
4. Named external evaluators — so the claim traces to someone.
5. Known limits — no limits section is an omission.

The card with no numbers

The teaching case is the card that says nothing measurable. Open-weights cards in wide use carry no quantitative safety section at all — benchmark scores, yes; no attack result at any attempt count. The packet records what exists: OMISSION — attack results not addressed; OMISSION — external evaluators not named.

An omission is a finding, recorded as such — not a gap to fill nor a mark against the vendor, but notice that the evidence must come from Meridian’s own evaluation and attack plan (Modules 3 and 4), or be accepted as absent by the signer.

Commercial practice, and the federal delta

The commercial starting practice is the third-party risk review, filed once per vendor — MU-AI-001’s already sits in R. Okafor’s files.

The federal delta is a named minimum. For the use case Meridian operates under the frame Lesson 1.4 declared — for a covered agency, with the federal terms written into the task order — OMB (the White House budget office that binds agencies) memorandum M-26-04, issued December 11, 2025, requires LLM vendors selling to federal agencies to disclose an acceptable-use policy, model, system, and data cards, end-user resources, and a feedback mechanism. Its enhanced tier — red-team evidence, training outside the United States by country, system prompts and content filters — is what an agency may require for a public-facing language model — an agency determination, not the high-impact category itself; requesting it from the vendor is the acquisition side’s move. The memo sunsets on December 11, 2027, so the packet records that date beside the checklist.

The handoff artifact is the packet with a column marking each M-26-04 item present, omitted, or not applicable. Not equivalent: a management-system certificate describes the vendor’s processes, not this model; a model card describes a family, not the deployment Meridian bought; and the list is a floor for federal language-model buys, not good practice for the outage classifier.

The gain is calendar time: a packet assembled in the federal shape from the start needs no second pass when the task order asks for it.

Contract facts you record, not negotiate

Five facts from the signed agreement, with their clauses: whether the vendor may train on Meridian’s data; what is retained and for how long, or whether retention is zero; the conditions on any indemnity — indemnities are conditioned on configurations — one large provider’s published indemnity, for example, is conditioned on its content filters staying on — so a team that turned them off may have turned off its indemnity; portability at exit; and the notice owed before a change or deprecation.

If a fact is absent, the packet says “absent” and this lesson stops. Handoff note: when Software Acquisition exists, negotiating absent terms, remedies and exit rights, and enhanced-tier disclosures as solicitation language migrate there; the five-fact record and the omission-as-finding rule stay.

An indemnity conditioned on a filter nobody checked is an indemnity Meridian does not have.

Stop and escalate when a vendor cannot supply any card, or the no-training or retention fact is absent for a system handling customer data — the packet records “absent,” a binding requirement that is absent goes to the acquisition side to cure, and whether the system keeps running meanwhile is the risk acceptor’s call, in writing.

Practice status — among organizations running vendor-hosted AI, commercial and federal

Scale: required | common baseline | strong optional | reference-shop (seen only at organizations that publish their own practice) | emerging

Key takeaway

Read from the risk acceptor’s seat, the packet holds claims, omissions, and applicability — never a verdict. Each absent document is recorded; a card with no attack curve yields an omission, not a paraphrase; an absent contract fact stops the lesson. The federal minimum list binds the federal-facing language-model procurement and is the reference shape for the rest. Lesson 2.2 turns to the version you can actually pin.

LEADERSHIP DECISION   fund reviewer hours to build the packet before
                      signature; treat an omission row as a finding
PRACTITIONER ACTION   quote claims, record omissions as "not
                      addressed", leave applicability to the owner
SUCCESS MEASURE       zero decisions signed with an unrecorded
                      omission; every indemnity condition checked
                      against the running configuration