A Triage Feature a Human Can Overrule
Updated
AdvancedWhat you'll learn
~55 min- Build a triage screen where a caseworker accepts, modifies or rejects an AI-drafted suggestion, labeled as AI-drafted in text and to assistive technology
- Store, for every suggestion, the deployment, version, prompt hash and time, and for every human decision, the actor, time, decision and reason
- Feed overridden cases into the next eval run, and keep a manual path that works with the model off
- Classify the feature against the federal definition of high-impact AI (OMB M-25-21, the memo governing federal agencies' AI use) without letting the policy become the lesson
What it is: the screen and the record behind one rule, that a model’s triage suggestion is a draft a named caseworker accepts, changes or rejects, with every step written down.
What it buys: a person who can overrule the machine in one click, a record that shows who did what and why, and a feedback loop where each overruled case is tested in the next evaluation.
What to fund: one caseworker-facing review of the screen for accessibility and wording, and an owner for the evaluation set.
Before the detail — Artifact: the triage panel, two append-only tables and an eval loader on the intake feature. Acceptance test: an override updates the record and appears in the next eval run, and with the model off a caseworker still completes triage (see Prove it). Status of what follows: reusable guidance, except the classification paragraph, which describes a federal memo and binds only the parties it names.
Lesson 26 produced a suggestion or a manual-path outcome. This lesson is everything after that: what the caseworker sees, what gets written, and what is deliberately absent. Nothing in this lesson calls submitIntake or touches an intake row’s status (the NEW or FAST_TRACK value lesson 22’s scoring sets); the only thing a decision changes is which queue and priority the case sits in, in the triage tables. Names from lessons 22 to 26: src/app/api/intake/ and src/features/intake/ (with submit.ts, score.ts and the SQL Server intakeStore in store.ts), the triage() service and its TriageOutcome type, and the data-provenance column data_source from lesson 10, which gains the value ai_drafted. Assumed, not defined by those lessons: a case page at /intake/cases/[caseId].
Plan the change with your agent
Start your agent in the project root and switch to Plan mode (Shift+Tab, /plan, or the mode picker in your desktop app). Ask it to plan the human-review layer for intake triage suggestions:
We need a human-review layer for intake triage suggestions: append-only suggestion and decision tables, a caseworker review panel, and an evaluation loader for overridden cases. Inspect src/features/intake/triage/ and ask me anything unclear before writing code.What to check in the plan before you approve
- Append-only database record and grants: Two SQL Server tables (
intake_triage_suggestionwithdata_source = 'ai_drafted', andintake_triage_decisionwithaccepted | modified | rejected | manual) where the app’s database identity is grantedSELECTandINSERTonly and explicitlyDENIEDUPDATEandDELETE. - Dual reason enforcement and session actor:
reasonis required wheneverdecisionismodifiedorrejectedin both a SQLCHECKconstraint and the Zod schema, andactoris always read from the authenticated session (session.sub), never from the request body. - Single-transaction object-level authorization:
decide()takes the whole session fromwithPermission, updates the case’s queue and priority with the case-visibility predicate (HOLDLOCKon the access table), and inserts the decision row in the same transaction, rolling back and returning404if zero rows match. - Accessible review panel and manual fallback:
TriagePanel.tsxlabels suggestions in visible text ("AI draft, not confirmed"), uses a keyboard-accessible radio group that reveals a required reason for Modify/Reject, moves focus to an alert region on error, and still completes manual triage when the model is disabled (suggestion === null). - Override evaluation loop:
loadEvalCases()returns the golden set plus the latest modified or rejected decision per case taggedsource: 'override', reported as its own slice rather than blended into the golden score.
If the first plan checks case visibility in a separate SELECT query before running the decision INSERT, correct it before approving:
Combine the case visibility check, the case queue/priority UPDATE, and the decision INSERT into a single SQL transaction that returns 404 when zero rows match.- Typed in the terminal:
claude - Mode:Plan mode
- You type in the session: We need a human-review layer for intake triage suggestions: append-only suggestion and decision tables, a caseworker review panel, and an evaluation loader for overridden cases. Inspect src/features/intake/triage/ and ask me anything unclear before writing code.
- The tool replies: I inspected src/features/intake/triage/. I will plan migrations for intake_triage_suggestion and intake_triage_decision, record.ts with Zod validation and decide(), TriagePanel.tsx, and eval-cases.ts. Should the tables enforce append-only via SQL DENY on UPDATE/DELETE, and require a reason on modified or rejected decisions?
- You type in the session: Yes. Deny UPDATE and DELETE for the app identity, enforce the reason in both SQL CHECK and Zod, and start with the record and tests.
- The tool replies: I'll write the record and its tests before any UI. Created migrations/0042_intake_triage_record.sql and 0043_intake_triage_grants.sql - two tables; the app identity gets SELECT and INSERT, with UPDATE and DELETE explicitly DENIED on both - CHECK: reason is required unless decision is accepted or manual - CHECK: data_source on suggestions is exactly 'ai_drafted' Created src/features/intake/triage/record.ts (Zod schemas + decide()) Created src/features/intake/triage/triage-review.test.ts Running vitest: 5 failed, as expected, because decide() and loadEvalCases() are stubs. A design point I want you to confirm: the app identity gets INSERT and SELECT only, and that is enforced by DENY, not by a comment. A correction to a decision is a new row that supersedes the old one, never an edit. Is that acceptable for your records schedule?
- You type in the session: Yes, supersede with a new row. One more thing: can a caseworker accept a suggestion for a case they don't have permission to see?
- The tool replies: No. decide() receives the whole session from withPermission('intake', 'update') and hands it to one store call that runs a single transaction. The first statement is an UPDATE of the case's queue and priority that carries the same visibility predicate the case page uses; if it does not affect exactly one row, the transaction rolls back and decide() returns 404, not 403, so the response does not confirm that the case exists. That is true of every decision, including a manual one. The decision row is inserted in the same transaction, so access revoked between the check and the write cannot be missed, and nothing is written on a 404. I added tests for it: 'decision on an invisible case returns 404 and writes nothing' for a suggestion-backed decision and for a manual one. Now the original four and the new ones pass: ✓ accepted decision stores actor, time, decision ✓ modified without a reason is refused (400), nothing written ✓ override reaches the next eval run as source: override ✓ model off: manual decision saves and updates the queue ✓ decision on an invisible case returns 404 and writes nothing ✓ a manual decision on an invisible case returns 404 and writes nothing The grants are proven separately by an environment test that connects as the app identity and expects UPDATE and DELETE to be refused (see Prove it). Next: TriagePanel.tsx. I'll render the AI label as text inside the section heading's description so a screen reader announces it before the suggested values.
- You type in the session: Build the panel. Then run the Playwright check with MODEL_ENABLED=false and tell me what the page contains.
- The tool replies: Created TriagePanel.tsx and e2e/intake-triage-manual.spec.ts. With MODEL_ENABLED=false the page contains: the heading 'Triage', the sentence 'No suggestion is available. Choose a queue and priority.', the two selects, and a Save button. The words 'AI draft, not confirmed' do not appear anywhere, because there is nothing AI-drafted to label. 1 passed (manual triage completes with the model off) With a suggestion present, the screen reader order is: heading, 'AI-drafted suggestion, not a decision', queue and priority, rationale, then the decision radio group.
Illustrative session; exact output varies by tool and agency configuration.
The record
Reference app’s choice: two append-only tables, one for what the machine said and one for what a person decided. Alternative: a single case-history table with a type column. Agency decision: your records schedule decides how long both are kept and whether a correction may ever rewrite a row. Contract test: the app’s database identity cannot update or delete either table.
REFERENCE CODE migrations/0042_intake_triage_record.sql: append-only suggestion and decision tables open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
-- migrations/0042_intake_triage_record.sqlCREATE TABLE intake_triage_suggestion ( id uniqueidentifier NOT NULL PRIMARY KEY, case_id nvarchar(64) NOT NULL, queue nvarchar(32) NOT NULL, priority nvarchar(16) NOT NULL, rationale nvarchar(500) NOT NULL, data_source varchar(20) NOT NULL CONSTRAINT ck_sugg_source CHECK (data_source = 'ai_drafted'), deployment nvarchar(128) NOT NULL, -- which deployment answered model_version nvarchar(128) NOT NULL, -- operator-declared version label prompt_hash char(64) NOT NULL, -- sha256 of the prompt version in lesson 26 created_at datetimeoffset NOT NULL);
CREATE TABLE intake_triage_decision ( id uniqueidentifier NOT NULL PRIMARY KEY, case_id nvarchar(64) NOT NULL, suggestion_id uniqueidentifier NULL REFERENCES intake_triage_suggestion(id), actor nvarchar(256) NOT NULL, -- from the session, never the request body decided_at datetimeoffset NOT NULL, decision varchar(10) NOT NULL CONSTRAINT ck_dec_kind CHECK (decision IN ('accepted','modified','rejected','manual')), final_queue nvarchar(32) NOT NULL, final_priority nvarchar(16) NOT NULL, reason nvarchar(1000) NULL, CONSTRAINT ck_dec_reason CHECK (decision IN ('accepted','manual') OR (reason IS NOT NULL AND LEN(LTRIM(RTRIM(reason))) > 0)));
-- Append-only is enforced by the grants in the next migration, not by this comment. REFERENCE CODE migrations/0043_intake_triage_grants.sql: SELECT/INSERT grants and UPDATE/DELETE denies open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
-- migrations/0043_intake_triage_grants.sql-- [intake-app-identity] stands for the app's database user (for an identity-token platform:-- CREATE USER [<identity name>] FROM EXTERNAL PROVIDER; for a stored secret, a plain login user). It must NOT be dbo, a member of-- db_owner, or a member of db_datawriter, because those memberships carry rights this file cannot take back.GRANT SELECT, INSERT ON OBJECT::dbo.intake_triage_suggestion TO [intake-app-identity];GRANT SELECT, INSERT ON OBJECT::dbo.intake_triage_decision TO [intake-app-identity];DENY UPDATE, DELETE ON OBJECT::dbo.intake_triage_suggestion TO [intake-app-identity];DENY UPDATE, DELETE ON OBJECT::dbo.intake_triage_decision TO [intake-app-identity];
-- recordDecision also sets the case's queue and priority in the same transaction. Grant exactly those columns-- (the table and column names here are assumptions; use your case table's):GRANT UPDATE (queue, priority) ON OBJECT::dbo.intake_case TO [intake-app-identity];
-- An UPDATE with a WHERE clause also needs SELECT on the table it reads, and the visibility-- predicate reads the access table. Grant reads on exactly what the batch touches. If an earlier-- migration (for the case page) already grants these, keep that one and drop these two lines.GRANT SELECT (case_id, queue, priority) ON OBJECT::dbo.intake_case TO [intake-app-identity];GRANT SELECT ON OBJECT::dbo.intake_case_access TO [intake-app-identity];The grants stop the application identity, which is the one a code defect or an injection would use. They do not stop a database administrator, who can still change a row. If your records schedule needs that closed too, an alternative is a ledger table (WITH (LEDGER = ON (APPEND_ONLY = ON)) in SQL Server, availability depends on your version, service and tier: verify it) in place of plain tables. Whichever you choose, the integration test under Prove it is the check that the identity really cannot rewrite history.
REFERENCE CODE src/features/intake/triage/record.ts: DecisionInput Zod schema and decide() service open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { z } from 'zod';import { QUEUES, PRIORITIES } from './schema';
export const DecisionInput = z .object({ caseId: z.string().min(1).max(64), suggestionId: z.string().uuid().nullable(), decision: z.enum(['accepted', 'modified', 'rejected', 'manual']), finalQueue: z.enum(QUEUES), finalPriority: z.enum(PRIORITIES), reason: z.string().trim().max(1000).optional(), }) .superRefine((v, ctx) => { const needsReason = v.decision === 'modified' || v.decision === 'rejected'; if (needsReason && !v.reason) ctx.addIssue({ code: 'custom', path: ['reason'], message: 'A reason is required' }); if (v.decision === 'manual' && v.suggestionId !== null) ctx.addIssue({ code: 'custom', path: ['suggestionId'], message: 'Manual triage has no suggestion' }); if (v.decision !== 'manual' && v.suggestionId === null) ctx.addIssue({ code: 'custom', path: ['suggestionId'], message: 'This decision needs a suggestion' }); });
export interface TriageSession { sub: string; roles: string[]; } // the object withPermission already built
export type RecordOutcome = 'recorded' | 'not_found' | 'accepted_mismatch' | 'modification_unchanged';
export interface TriageStore { // ONE transaction (see the T-SQL below): the case UPDATE carries the visibility predicate and must affect // exactly one row, the suggestion check and the decision INSERT follow in the same transaction. // 'not_found' covers a case that does not exist, one the session may not see, and a suggestion that is not // this case's; callers cannot tell which. Nothing is written for any outcome except 'recorded'. recordDecision(session: TriageSession, d: { id: string; caseId: string; suggestionId: string | null; actor: string; decidedAt: string; decision: string; finalQueue: string; finalPriority: string; reason?: string; }): Promise<RecordOutcome>; listDecisions(): Promise<Array<{ caseId: string; decidedAt: string; decision: string; finalQueue: string; finalPriority: string }>>;}
export async function decide( session: TriageSession, // the whole session; the actor is session.sub raw: unknown, deps: { store: TriageStore; now: () => Date; newId: () => string },): Promise<{ ok: true } | { ok: false; status: 400 | 404; message: string }> { const parsed = DecisionInput.safeParse(raw); if (!parsed.success) return { ok: false, status: 400, message: 'Invalid decision' }; const d = parsed.data;
// There is deliberately no separate "can this session see the case?" call before this one. A check and a write // in two operations leave a gap in which access can be revoked; the store does both in one transaction. const outcome = await deps.store.recordDecision(session, { id: deps.newId(), caseId: d.caseId, suggestionId: d.suggestionId, actor: session.sub, decidedAt: deps.now().toISOString(), decision: d.decision, finalQueue: d.finalQueue, finalPriority: d.finalPriority, reason: d.reason, }); switch (outcome) { case 'recorded': return { ok: true }; case 'not_found': return { ok: false, status: 404, message: 'Not found' }; // same answer for "missing" and "not yours" case 'accepted_mismatch': return { ok: false, status: 400, message: 'Accepted values must match the suggestion' }; case 'modification_unchanged': return { ok: false, status: 400, message: 'A modification must change something' }; }}The reference route in gov-dev-reference-app passes the whole session from withPermission into decide (for comparison with what your agent writes; you do not type it by hand):
export const POST = withPermission('intake', 'update', async (req, session) => { const out = await decide(session, await req.json().catch(() => null), { store: triageStore, now: () => new Date(), newId: () => crypto.randomUUID() }); return out.ok ? new Response(null, { status: 204 }) : errorResponse(out.message, out.status);});withPermission says the caller may update triage in general; the visibility predicate says they may update this case. A role check alone is function-level authorization and leaves every case id guessable by any caseworker. Checking visibility in one call and writing in another is still not enough: access revoked between the two calls would be missed. So the check is part of the write.
Isolation. The transaction runs at READ COMMITTED, the SQL Server default. Two things make it honor a revocation. The UPDATE ... WHERE <visibility predicate> locks the case row until commit. The read of the access table carries HOLDLOCK (the SERIALIZABLE hint for that one read), which holds a range lock on the matching access rows until commit, so a concurrent revoke waits for this transaction and then takes effect, and the next decision gets a 404. Index the access table on (case_id, principal_sub) so the range is one row. A deadlock (error 1205) can occur if a revoke takes its locks in the other order; the store retries once. The predicate is a sketch: it must be the one your case page already uses (the assignment table is an assumption), kept in one shared place so both paths mean the same thing.
REFERENCE CODE src/features/intake/triage/record-decision.sql: single-transaction visibility check, case update, and decision insert open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
-- src/features/intake/triage/record-decision.sql (sent as one batch by sql-store.ts)SET XACT_ABORT ON; -- any runtime error rolls the whole transaction backBEGIN TRANSACTION; -- READ COMMITTED; the HOLDLOCK below is the only stronger read
UPDATE c SET c.queue = @finalQueue, c.priority = @finalPriority FROM dbo.intake_case AS c WHERE c.case_id = @caseId AND EXISTS (SELECT 1 FROM dbo.intake_case_access AS a WITH (HOLDLOCK) WHERE a.case_id = c.case_id AND a.principal_sub = @sub);
IF @@ROWCOUNT <> 1 -- missing, or not visible to this session, or revoked a moment agoBEGIN ROLLBACK TRANSACTION; SELECT 'not_found' AS outcome; RETURN;END;
IF @suggestionId IS NOT NULLBEGIN DECLARE @sq nvarchar(32), @sp nvarchar(16); SELECT @sq = queue, @sp = priority FROM dbo.intake_triage_suggestion WHERE id = @suggestionId AND case_id = @caseId; IF @@ROWCOUNT <> 1 BEGIN ROLLBACK TRANSACTION; SELECT 'not_found' AS outcome; RETURN; END; IF @decision = 'accepted' AND (@sq <> @finalQueue OR @sp <> @finalPriority) BEGIN ROLLBACK TRANSACTION; SELECT 'accepted_mismatch' AS outcome; RETURN; END; IF @decision = 'modified' AND @sq = @finalQueue AND @sp = @finalPriority BEGIN ROLLBACK TRANSACTION; SELECT 'modification_unchanged' AS outcome; RETURN; END;END;
INSERT INTO dbo.intake_triage_decision (id, case_id, suggestion_id, actor, decided_at, decision, final_queue, final_priority, reason)VALUES (@id, @caseId, @suggestionId, @actor, @decidedAt, @decision, @finalQueue, @finalPriority, @reason);
COMMIT TRANSACTION;SELECT 'recorded' AS outcome; REFERENCE CODE src/features/intake/triage/sql-store.ts: SQL Server store executing record-decision.sql with deadlock retry open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
// src/features/intake/triage/sql-store.ts (the batch above, via the lesson 08 pool)import sql from 'mssql';import { readFileSync } from 'node:fs';import type { RecordOutcome, TriageSession, TriageStore } from './record';
const BATCH = readFileSync(new URL('./record-decision.sql', import.meta.url), 'utf8');
export function sqlTriageRecorder(pool: sql.ConnectionPool): TriageStore['recordDecision'] { const run = async (session: TriageSession, d: Parameters<TriageStore['recordDecision']>[1]): Promise<RecordOutcome> => { const r = await pool.request() .input('sub', sql.NVarChar(256), session.sub) .input('id', sql.UniqueIdentifier, d.id) .input('caseId', sql.NVarChar(64), d.caseId) .input('suggestionId', sql.UniqueIdentifier, d.suggestionId) .input('actor', sql.NVarChar(256), d.actor) .input('decidedAt', sql.DateTimeOffset, new Date(d.decidedAt)) .input('decision', sql.VarChar(10), d.decision) .input('finalQueue', sql.NVarChar(32), d.finalQueue) .input('finalPriority', sql.NVarChar(16), d.finalPriority) .input('reason', sql.NVarChar(1000), d.reason ?? null) .query(BATCH); return r.recordset[0].outcome as RecordOutcome; }; return async (session, d) => { try { return await run(session, d); } catch (e: any) { if (e?.number === 1205) return run(session, d); throw e; } // deadlock victim: retry once };}The suggestion is saved at the moment lesson 26’s triage() returns suggested, using its meta for deployment, version, prompt hash and time. The field that tells later readers a value was machine-drafted is data_source = 'ai_drafted', the same provenance rule as lesson 10: never displayed as an observed fact, never merged into a column that holds recorded ones.
Nothing in this module sends a notice, closes a case or changes eligibility. Fresh authorization for each consequential action, and a receipt the acting system cannot write, is the job of Guarded Automation (one action, one moment, a separate training in this series); if intake ever adds an automatic action, that is the design to adopt first.
The screen
REFERENCE CODE src/features/intake/triage/TriagePanel.tsx: accessible caseworker review panel with manual fallback open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
'use client';import { useId, useRef, useState } from 'react';import { QUEUES, PRIORITIES } from './schema';
type Suggestion = { id: string; queue: string; priority: string; rationale: string } | null;type Choice = 'accepted' | 'modified' | 'rejected';
export function TriagePanel({ caseId, suggestion }: { caseId: string; suggestion: Suggestion }) { const uid = useId(); const errorRef = useRef<HTMLDivElement>(null); const [choice, setChoice] = useState<Choice>('accepted'); const [queue, setQueue] = useState(suggestion?.queue ?? QUEUES[0]); const [priority, setPriority] = useState(suggestion?.priority ?? PRIORITIES[0]); const [reason, setReason] = useState(''); const [error, setError] = useState(''); const [saved, setSaved] = useState(false);
const manual = suggestion === null; const needsReason = !manual && choice !== 'accepted'; const editable = manual || choice !== 'accepted';
async function submit(e: React.FormEvent) { e.preventDefault(); setError(''); if (needsReason && !reason.trim()) { setError('Enter a reason before saving.'); queueMicrotask(() => errorRef.current?.focus()); return; } const res = await fetch('/api/intake/triage/decision', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ caseId, suggestionId: suggestion?.id ?? null, decision: manual ? 'manual' : choice, finalQueue: choice === 'accepted' && suggestion ? suggestion.queue : queue, finalPriority: choice === 'accepted' && suggestion ? suggestion.priority : priority, reason: reason.trim() || undefined, }), }); if (!res.ok) { setError('The decision was not saved. Nothing has changed. Try again, or contact your supervisor.'); queueMicrotask(() => errorRef.current?.focus()); return; } setSaved(true); }
return ( <section aria-labelledby={`${uid}-h`} aria-describedby={suggestion ? `${uid}-ai` : undefined}> <h2 id={`${uid}-h`}>Triage</h2>
{suggestion ? ( <div id={`${uid}-ai`}> <p><strong>AI draft, not confirmed.</strong> This is a suggestion, not a decision: a person must accept, change or reject it.</p> <dl> <dt>Queue</dt><dd>{suggestion.queue}</dd> <dt>Priority</dt><dd>{suggestion.priority}</dd> <dt>Model's stated reason</dt><dd>{suggestion.rationale}</dd> </dl> </div> ) : ( <p>No suggestion is available. Choose a queue and priority.</p> )}
<form onSubmit={submit}> {!manual && ( <fieldset> <legend>Your decision</legend> {(['accepted', 'modified', 'rejected'] as const).map((c) => ( <label key={c} className="block"> <input type="radio" name={`${uid}-choice`} checked={choice === c} onChange={() => setChoice(c)} />{' '} {c === 'accepted' ? 'Accept' : c === 'modified' ? 'Modify' : 'Reject'} </label> ))} </fieldset> )}
<label>Queue{' '} <select value={queue} disabled={!editable} onChange={(e) => setQueue(e.target.value)}> {QUEUES.map((q) => <option key={q}>{q}</option>)} </select> </label> <label>Priority{' '} <select value={priority} disabled={!editable} onChange={(e) => setPriority(e.target.value)}> {PRIORITIES.map((p) => <option key={p}>{p}</option>)} </select> </label>
{needsReason && ( <label className="block">Reason (required) <textarea value={reason} onChange={(e) => setReason(e.target.value)} aria-required="true" aria-describedby={error ? `${uid}-err` : undefined} /> </label> )}
<div ref={errorRef} tabIndex={-1} role="alert" id={`${uid}-err`}>{error}</div> <button type="submit">Save triage</button> </form>
<div role="status">{saved ? 'Triage saved.' : ''}</div> </section> );}The panel uses Tailwind classes, not style={{ ... }}, on purpose. Lesson 4’s policy sets style-src 'self' 'nonce-<value>', which allows <style> elements that carry the nonce and blocks every style="" attribute, so React’s inline style prop is refused in production. Tailwind utilities arrive in the stylesheet. If you swap in MUI components, style them with sx: Emotion writes sx as <style> elements, and lesson 4 wires the nonce to them through <AppRouterCacheProvider options={{ nonce, enableCssLayer: true }}>, so they pass. Do not reach for the style prop on either kind of component.
Swap the native controls for your lesson 11 components; keep the structure: a text label that does not rely on color, a labeled radio group, a required reason that appears only when it is needed, an alert region that receives focus, and a status region for success. Lesson 14’s audit will check these against the accessibility standard your agency is held to.
The loop: overrides become tests
REFERENCE CODE src/features/intake/triage/eval-cases.ts: load golden cases plus the latest overridden decisions open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import type { TriageStore } from './record';
export interface EvalCase { caseId: string; expected: { queue: string; priority: string }; source: 'golden' | 'override'; reviewed: boolean; // an override is one caseworker's judgment until a second person confirms it}
export async function loadEvalCases(deps: { store: TriageStore; golden: EvalCase[] }): Promise<EvalCase[]> { const goldenIds = new Set(deps.golden.map((c) => c.caseId)); const latest = new Map<string, Awaited<ReturnType<TriageStore['listDecisions']>>[number]>(); for (const d of await deps.store.listDecisions()) { const prev = latest.get(d.caseId); if (!prev || prev.decidedAt < d.decidedAt) latest.set(d.caseId, d); } const overrides: EvalCase[] = []; for (const d of latest.values()) { if ((d.decision === 'modified' || d.decision === 'rejected') && !goldenIds.has(d.caseId)) { overrides.push({ caseId: d.caseId, expected: { queue: d.finalQueue, priority: d.finalPriority }, source: 'override', reviewed: false }); } } return [...deps.golden, ...overrides];}An eval case stores the case id and the expected answer, not a copy of the narrative; the runner resolves the narrative from the case store at run time, inside the boundary, so the eval set does not become a second copy of sensitive text. The override slice is reported on its own line, not folded into the golden score: it is the set of cases the model has already got wrong once, so its rate answers a different question. A reviewed flag keeps one person’s disagreement from silently becoming ground truth. How to design and gate an evaluation is AI Assurance’s subject; its consequence tier and gate path (a separate training in this series) is where the tier that sets how strict the release gate must be gets decided, and the release gate is where a changed model or prompt hash is held until the eval passes.
Does it fall under the federal high-impact AI definition?
For federal agencies, and for contractors only where a contract carries the term, OMB M-25-21 (the April 2025 memo governing federal use of AI) defines high-impact AI as AI “with an output that serves as a principal basis for decisions or actions with legal, material, binding, or significant effect” on, among other things, an individual’s civil rights, civil liberties or privacy, or their access to “critical government resources or services”. If a use case meets that definition, the memo’s section 4(b) requires seven minimum practices for it, under these headings: “Conduct Pre-Deployment Testing”; “Complete AI Impact Assessment”; “Conduct Ongoing Monitoring for Performance and Potential Adverse Impacts”; “Ensure Adequate Human Training and Assessment”; “Provide Additional Human Oversight, Intervention, and Accountability”; “Offer Consistent Remedies or Appeals”; and “Consult and Incorporate Feedback from End Users and the Public”. Documented implementation was due 365 days after the memo (April 3, 2026 by our count), a high-impact use that does not comply must be safely discontinued, and pilots are exempt. Separately, every federal agency except the Department of Defense and the Intelligence Community must inventory its AI use cases at least annually. Whether intake triage meets the definition turns on the phrase “principal basis”: the memo’s own footnote 27 says AI can sit inside a high-impact process without meeting it if its output is not a principal basis. A queue and priority suggestion that a caseworker reviews is arguably not one; a screen where the suggestion is accepted unread on eligibility-affecting cases could become one. That determination, and the inventory entry, belong to your agency’s AI governance owner, not to this code. What this lesson’s record gives that owner is evidence about the human oversight and the testing, not a classification. State and local readers: the memo does not govern you; use your own agency’s policy.
Availability, authorization, agency approval, region support, and price are five separate facts. This lesson states none of them for any model service. The M-25-21 wording above was read from the memo itself (https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf): the high-impact definition in its definitions section, the seven minimum practices in section 4(b), and the inventory duty in section 3(b)(v). Not checked here: whether a later OMB memo superseded it, and which version of the annual inventory instructions your agency currently follows. Recheck the cited source and your agency’s authorization boundary before you rely on any of it.
Prove it
Six behaviors: the override loop, the model-off path, object-level authorization, the revoke race, the append-only grants, and the page under the production CSP. The first three run with no database.
REFERENCE TEST src/features/intake/triage/triage-review.test.ts: unit tests for overrides, object-level auth, and model-off fallback open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { describe, it, expect } from 'vitest';import { decide, type TriageStore } from './record';import { loadEvalCases } from './eval-cases';import { triage } from './service';
const SESSION = { sub: 'r.caseworker', roles: ['intake.caseworker'] };const VISIBLE_CASES = new Set(['C-9', 'C-10']); // C-77 exists but this session cannot see itconst SUGGESTED = { queue: 'general', priority: 'routine' };
// Mirrors the T-SQL batch's outcomes: the visibility check and the write are one step, and nothing is// written unless the outcome is 'recorded'. (The real transaction is proven by the live race test below.)function memoryStore(): TriageStore & { rows: any[]; queueOf: Map<string, string> } { const rows: any[] = []; const queueOf = new Map<string, string>(); const known = new Set(['11111111-1111-4111-8111-111111111111']); return { rows, queueOf, async recordDecision(_session, d) { if (!VISIBLE_CASES.has(d.caseId)) return 'not_found'; if (d.suggestionId) { if (!known.has(d.suggestionId)) return 'not_found'; const same = d.finalQueue === SUGGESTED.queue && d.finalPriority === SUGGESTED.priority; if (d.decision === 'accepted' && !same) return 'accepted_mismatch'; if (d.decision === 'modified' && same) return 'modification_unchanged'; } rows.push(d); queueOf.set(d.caseId, `${d.finalQueue}/${d.finalPriority}`); return 'recorded'; }, async listDecisions() { return rows; }, };}const SID = '11111111-1111-4111-8111-111111111111';let n = 0;const clock = (iso: string) => () => new Date(iso);const ids = () => `id-${++n}`;
describe('override reaches the record and the next eval', () => { it('a modified decision stores actor, time, decision and reason, and updates the case', async () => { const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-9', suggestionId: SID, decision: 'modified', finalQueue: 'escalations', finalPriority: 'urgent', reason: 'Threat of eviction in writing' }, { store, now: clock('2026-11-04T12:00:00Z'), newId: ids }); expect(r).toEqual({ ok: true }); expect(store.rows[0]).toMatchObject({ actor: 'r.caseworker', decidedAt: '2026-11-04T12:00:00.000Z', decision: 'modified', reason: 'Threat of eviction in writing' }); expect(store.queueOf.get('C-9')).toBe('escalations/urgent');
const cases = await loadEvalCases({ store, golden: [] }); expect(cases).toContainEqual(expect.objectContaining({ caseId: 'C-9', source: 'override', expected: { queue: 'escalations', priority: 'urgent' } })); });
it('a modification without a reason is refused and nothing is written', async () => { const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-9', suggestionId: SID, decision: 'rejected', finalQueue: 'records', finalPriority: 'routine' }, { store, now: clock('2026-11-04T12:00:00Z'), newId: ids }); expect(r).toMatchObject({ ok: false, status: 400 }); expect(store.rows).toHaveLength(0); });});
describe('object-level authorization', () => { it('decision on an invisible case returns 404 and writes nothing', async () => { const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-77', suggestionId: SID, decision: 'modified', finalQueue: 'escalations', finalPriority: 'urgent', reason: 'Probing another office' }, { store, now: clock('2026-11-04T12:00:00Z'), newId: ids }); expect(r).toEqual({ ok: false, status: 404, message: 'Not found' }); expect(store.rows).toHaveLength(0); expect(store.queueOf.size).toBe(0); });
it('a manual decision on an invisible case returns 404 and writes nothing', async () => { const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-77', suggestionId: null, decision: 'manual', finalQueue: 'general', finalPriority: 'routine' }, { store, now: clock('2026-11-04T12:00:00Z'), newId: ids }); expect(r).toEqual({ ok: false, status: 404, message: 'Not found' }); expect(store.rows).toHaveLength(0); expect(store.queueOf.size).toBe(0); });
it('a case that does not exist gets the same answer as one that is not visible', async () => { const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-DOES-NOT-EXIST', suggestionId: null, decision: 'manual', finalQueue: 'general', finalPriority: 'routine' }, { store, now: clock('2026-11-04T12:00:00Z'), newId: ids }); expect(r).toEqual({ ok: false, status: 404, message: 'Not found' }); });});
describe('model off', () => { it('triage reports manual, and a manual decision still saves', async () => { const out = await triage({ caseId: 'C-10', narrative: 'help' }, { provider: null, log: () => {}, now: () => new Date() }); expect(out).toEqual({ status: 'manual', reason: 'model_disabled' });
const store = memoryStore(); const r = await decide(SESSION, { caseId: 'C-10', suggestionId: null, decision: 'manual', finalQueue: 'general', finalPriority: 'routine' }, { store, now: clock('2026-11-04T12:05:00Z'), newId: ids }); expect(r).toEqual({ ok: true }); expect(store.queueOf.get('C-10')).toBe('general/routine'); });});The reference Playwright test in e2e/intake-triage-manual.spec.ts from gov-dev-reference-app verifies the manual fallback when MODEL_ENABLED=false (compare with what your agent writes; you do not type it by hand):
// e2e/intake-triage-manual.spec.ts (run with the app started with MODEL_ENABLED=false)import { test, expect } from '@playwright/test';
test('manual triage completes with the model off', async ({ page }) => { await page.goto('/intake/cases/C-10'); await expect(page.getByText('AI draft, not confirmed')).toHaveCount(0); await page.getByLabel('Queue').selectOption('records'); await page.getByLabel('Priority').selectOption('elevated'); await page.getByRole('button', { name: 'Save triage' }).click(); await expect(page.getByRole('status')).toHaveText('Triage saved.');});The inline-style defect shows only under the production policy, so the reference CSP test in e2e/intake-triage-csp.spec.ts from gov-dev-reference-app runs against next build then next start (dev mode adds directives and proves less), the same way lesson 4’s nonce test does:
// e2e/intake-triage-csp.spec.ts (run against a production build)import { test, expect } from '@playwright/test';
test('the triage page loads under the production CSP with no violation', async ({ page }) => { const refused: string[] = []; page.on('console', (m) => { if (/Refused to apply inline style|violates the following Content Security Policy/.test(m.text())) refused.push(m.text()); }); await page.goto('/intake/cases/C-10'); await expect(page.getByRole('heading', { name: 'Triage' })).toBeVisible(); expect(refused, 'no CSP violation reported in the console').toEqual([]); expect(await page.locator('section [style]').count(), 'no style="" attribute inside the panel').toBe(0);});The grants are tested against a real database as the app identity, because a mock cannot prove a permission. This is an environment test, so it carries the .live.test.ts suffix that lesson 38’s cold run excludes and gates behind INTAKE_LIVE=1; point it at a test database, since append-only means its rows stay.
REFERENCE TEST src/features/intake/triage/triage-grants.live.test.ts: live database verification of append-only grants open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { describe, it, expect, beforeAll, afterAll } from 'vitest';import { randomUUID } from 'node:crypto';import sql from 'mssql';
// TRIAGE_DB_APP_CONNECTION is a connection string for the APP identity (not an admin) on a TEST database.const conn = process.env.TRIAGE_DB_APP_CONNECTION;const live = process.env.INTAKE_LIVE === '1';
describe.skipIf(!live)('LIVE: the app identity cannot rewrite the triage record', () => { let pool: sql.ConnectionPool; const suggestionId = randomUUID(); const decisionId = randomUUID();
beforeAll(async () => { expect(conn, 'TRIAGE_DB_APP_CONNECTION must be set for the live grant test').toBeTruthy(); pool = await new sql.ConnectionPool(conn!).connect(); }); afterAll(async () => { await pool?.close(); });
it('can INSERT and SELECT on both tables', async () => { await pool.request().input('id', sql.UniqueIdentifier, suggestionId).query( `INSERT INTO dbo.intake_triage_suggestion (id, case_id, queue, priority, rationale, data_source, deployment, model_version, prompt_hash, created_at) VALUES (@id, 'GRANT-TEST', 'general', 'routine', 'grant test', 'ai_drafted', 'test', 'test', REPLICATE('a', 64), SYSDATETIMEOFFSET())`); await pool.request().input('id', sql.UniqueIdentifier, decisionId).input('sid', sql.UniqueIdentifier, suggestionId).query( `INSERT INTO dbo.intake_triage_decision (id, case_id, suggestion_id, actor, decided_at, decision, final_queue, final_priority) VALUES (@id, 'GRANT-TEST', @sid, 'grant-test', SYSDATETIMEOFFSET(), 'accepted', 'general', 'routine')`); const r = await pool.request().input('id', sql.UniqueIdentifier, decisionId).query(`SELECT id FROM dbo.intake_triage_decision WHERE id = @id`); expect(r.recordset).toHaveLength(1); });
// 229 is SQL Server's "permission was denied on the object" error. it.each([ ['UPDATE suggestion', `UPDATE dbo.intake_triage_suggestion SET rationale = 'rewritten' WHERE id = @id`, 'suggestion'], ['DELETE suggestion', `DELETE FROM dbo.intake_triage_suggestion WHERE id = @id`, 'suggestion'], ['UPDATE decision', `UPDATE dbo.intake_triage_decision SET reason = 'rewritten' WHERE id = @id`, 'decision'], ['DELETE decision', `DELETE FROM dbo.intake_triage_decision WHERE id = @id`, 'decision'], ])('%s is refused', async (_name, stmt, which) => { const id = which === 'suggestion' ? suggestionId : decisionId; await expect(pool.request().input('id', sql.UniqueIdentifier, id).query(stmt)).rejects.toMatchObject({ number: 229 }); });
it('TRUNCATE is refused', async () => { await expect(pool.request().query(`TRUNCATE TABLE dbo.intake_triage_decision`)).rejects.toThrow(); });});The race is also proven against a real database, because only the database can show that a revoke and a decision serialize. A second, privileged connection revokes access inside an open transaction while the decision is in flight; the decision must wait, then find the access gone.
REFERENCE TEST src/features/intake/triage/triage-race.live.test.ts: live database verification of mid-request access revocation open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { describe, it, expect, beforeAll, afterAll } from 'vitest';import { randomUUID } from 'node:crypto';import sql from 'mssql';import { sqlTriageRecorder } from './sql-store';
// APP = the app identity; ADMIN = a test-only privileged connection that seeds and revokes. Both on a TEST database.const live = process.env.INTAKE_LIVE === '1';describe.skipIf(!live)('LIVE: a revoke that lands before the decision commits is honored', () => { let app: sql.ConnectionPool, admin: sql.ConnectionPool; const caseId = `RACE-${randomUUID().slice(0, 8)}`; const sub = 'race-caseworker';
beforeAll(async () => { app = await new sql.ConnectionPool(process.env.TRIAGE_DB_APP_CONNECTION!).connect(); admin = await new sql.ConnectionPool(process.env.TRIAGE_DB_ADMIN_CONNECTION!).connect(); // table and column names are assumptions; use your case and access tables await admin.request().input('c', caseId).query(`INSERT INTO dbo.intake_case (case_id, queue, priority) VALUES (@c, 'general', 'routine')`); await admin.request().input('c', caseId).input('o', sub).query(`INSERT INTO dbo.intake_case_access (case_id, principal_sub) VALUES (@c, @o)`); }); afterAll(async () => { await app?.close(); await admin?.close(); });
it('returns not_found and writes nothing when access is revoked mid-request', async () => { const revoke = new sql.Transaction(admin); await revoke.begin(); // the revoke is open, not yet committed await revoke.request().input('c', caseId).query(`DELETE FROM dbo.intake_case_access WHERE case_id = @c`);
const record = sqlTriageRecorder(app); const id = randomUUID(); const pending = record({ sub, roles: [] }, { id, caseId, suggestionId: null, actor: sub, decidedAt: new Date().toISOString(), decision: 'manual', finalQueue: 'records', finalPriority: 'urgent', });
const early = await Promise.race([pending, new Promise((r) => setTimeout(() => r('still waiting'), 750))]); expect(early, 'the decision waits for the revoke instead of reading stale access').toBe('still waiting'); await revoke.commit();
expect(await pending).toBe('not_found'); const rows = await admin.request().input('i', sql.UniqueIdentifier, id).query(`SELECT 1 AS n FROM dbo.intake_triage_decision WHERE id = @i`); expect(rows.recordset).toHaveLength(0); const c = await admin.request().input('c', caseId).query(`SELECT queue, priority FROM dbo.intake_case WHERE case_id = @c`); expect(c.recordset[0]).toEqual({ queue: 'general', priority: 'routine' }); });});npx vitest run src/features/intake/triage/triage-review.test.tsINTAKE_LIVE=1 TRIAGE_DB_APP_CONNECTION='<test database, app identity>' TRIAGE_DB_ADMIN_CONNECTION='<test database, seeding admin>' \ npx vitest run src/features/intake/triage/triage-grants.live.test.ts src/features/intake/triage/triage-race.live.test.tsMODEL_ENABLED=false npx playwright test e2e/intake-triage-manual.spec.tsnpm run build && npm run start & # then, with the production server up:npx playwright test e2e/intake-triage-csp.spec.ts# The only state change lessons 22-23 define is submitIntake setting status NEW / FAST_TRACK.# Nothing in the triage folder may call it, import the intake store, or write a status:! grep -rEn "submitIntake|insertIntake|intakeStore|from '\.\./(submit|store)'|FAST_TRACK|status *=" src/features/intake/triage --include='*.ts' --include='*.tsx' --exclude='*.test.ts'Pass: Vitest green, both Playwright specs green (the CSP spec reports no “Refused to apply inline style” and no CSP violation), and the grep prints nothing. A suggestion never calls submitIntake and never writes status. The negative cases to plant: remove the reason rule from DecisionInput and the “refused” test must go red; delete the visibility predicate from the batch’s UPDATE (or the HOLDLOCK hint) and the live race test must go red, and a store that returns recorded for an invisible case turns both “invisible case” unit tests red; remove one DENY line from the grants migration (or add the identity to db_datawriter) and the matching row of the live grant test must go red; put a style={{ display: 'block' }} back on a label and the CSP spec must go red; add an import of submitIntake into record.ts and the grep must print the line and the command must fail.
A caseworker modifies a suggestion and saves. Which record is correct?
Why is the override slice of the eval run reported separately from the golden set?
Practice status
| Practice | Status | Also called |
|---|---|---|
| Suggestion shown as a draft a person accepts, changes or rejects | common baseline | human-in-the-loop, human review step |
| Suggestion labeled as AI-drafted in text and to assistive technology | common baseline | AI disclosure label, provenance badge |
| Append-only record of suggestion and of each human decision | strong optional | decision log, audit trail |
| Model/deployment, version and prompt hash stored with each suggestion | strong optional | model card reference, inference record |
| Overridden cases fed into the next eval run | strong optional | feedback loop, regression set from corrections |
| Manual path that works with the model off | common baseline | graceful degradation, business-continuity path |
| Annual AI use case inventory entry | required | AI use case inventory, system-of-record entry |
| Agency-owned determination against the M-25-21 high-impact definition | required | high-impact determination, impact classification |
| Object-level authorization on every decision, manual included | common baseline | per-record access check, insecure direct object reference prevention |
| Append-only record enforced by database grants, not by convention | strong optional | write-once audit table, ledger table |
Scale: required | common baseline | strong optional | reference-shop (seen only at organizations that publish their own practice) | emerging
Note: “required” in the two M-25-21 rows applies to federal agencies under the memo (the inventory duty excludes the Department of Defense and the Intelligence Community); everyone else follows their own agency’s policy. The determination is the agency’s, not the developer’s.
Key takeaway
- The suggestion and the decision are two records with two authors; neither overwrites the other.
- Nothing consequential happens automatically. The only effect of a decision is the case’s queue and priority.
- Overrides are the cheapest labeled data the office will ever get, if they are kept apart until someone reviews them.
- Classification is the agency’s call. The record supplies evidence for it and does not make it.
- LEADERSHIP DECISION
- Who owns the high-impact determination and the use case inventory entry, and who owns the eval set.
- PRACTITIONER ACTION
- Ship the two tables, the panel and the eval loader; run the model-off Playwright check before each release.
- SUCCESS MEASURE
- Audit finding avoided: every triage on a sampled case traces to a named person, a reason and, where a suggestion existed, a prompt hash.
Legend: LEADERSHIP DECISION = what a leader decides · PRACTITIONER ACTION = what the engineer does next · SUCCESS MEASURE = how you know, in calendar time, assessor-hours or cost, contract or revenue risk, or audit finding avoided