Strangle It With Differential Tests
Updated
IntermediateWhat you'll learn
~55 min- Route one legacy route to the new intake feature through a façade (a thin routing layer in front of both systems) with a one-setting rollback
- Build a differential harness that replays the golden corpus through old and new and compares every output and side effect under written, field-specific tolerances
- Report coverage by rule from rules.md, so a mismatch names the business rule it broke
- Plant a rounding change in the new code and watch the harness block it
Lesson 22 ended with three things for the case-intake office: rules.md (seven rules, each cited), unknowns.md, and a ten-case golden corpus recorded from a restored copy of the legacy app. This lesson builds the intake feature on the reference app to match that corpus, puts a façade in front so one route moves while everything else stays legacy, and proves the match with a harness that replays the corpus through both and refuses to pass on a mismatch.
Before the detail — Artifact: a routing façade, a differential harness with a tolerances file, and a rule-coverage report. Acceptance test: a deliberate rounding change in the new priority score fails the harness and names the rule it broke, R-04. Status: reusable guidance; the pattern is Microsoft’s published Strangler Fig, the file layout and names are this module’s.
The pattern, with its sources
Microsoft’s Azure Architecture Center describes the Strangler Fig pattern as incrementally migrating a legacy system by gradually replacing specific pieces of functionality with new applications and services; the page shows “Last updated on 2026-06-02” (Strangler Fig pattern). A façade sits in front, sends each request to old or new, and the old system shrinks as routes move. You never run a big-bang cutover, and you can stop at any point with a working system.
The test side is characterization testing, which Michael C. Feathers set out in Working Effectively with Legacy Code (Prentice Hall PTR, 2004): write tests that pin down what the code actually does today, bugs included, before you change it. The same idea travels under other names, and you will meet them in vendor and audit documents:
- golden master or approval test — recorded outputs approved as the reference, compared on every run (lesson 22’s corpus).
- shadow mode or parallel run — real traffic goes to both systems and only the legacy answer is used; the new answer is compared and discarded.
- differential test — this lesson’s term: the same inputs through old and new, diff the outputs.
Cloud Modernization Patterns (a separate training in this series) lesson 5.2 already teaches the discipline underneath: preserve behavior first, list every legitimate difference explicitly, improve later as ordinary work. This lesson is the hands-on build of the harness for a classic ASP and stored-procedure app. The corpus, rules.md and rule IDs come from lesson 22.
What gets built, and where
| Path | What it is |
|---|---|
src/features/intake/score.ts | priorityScore() and bankersRound(): rules R-01 to R-05 |
src/features/intake/submit.ts | submitIntake(): rules R-06 and R-07, writes through an IntakeStore |
src/features/intake/store.ts | intakeStore: the SQL Server implementation of IntakeStore, reaching the database through withPool (lesson 08’s connection pattern) |
src/app/api/intake/route.ts | POST /api/intake: same form fields and same plain-text answer as submit.asp |
src/features/intake/facade.ts | routeFor(): the pure routing decision |
src/proxy.ts | applies routeFor() to /intake/:path* (coarse routing only, not authorization) |
tests/differential/ | harness.ts, tolerances.ts, intake.diff.test.ts, tolerances.test.ts, facade.test.ts |
The new route accepts the legacy request shape and answers in the legacy format on purpose. While two systems share one URL space, a caller cannot tell which answered; a cleaner API shape is a later, separate change.
Step 1: the feature, written from rules.md
Start in Plan mode (Shift+Tab, /plan, or the mode picker) so the agent builds from modernization/rules.md rather than reading the old ASP directly:
Read modernization/rules.md and modernization/unknowns.md, and plan the new intake feature in src/features/intake/ so it reproduces rules R-01 through R-07 without touching modernization/legacy/. Ask me anything unclear before writing code.Before approving the feature plan, verify that it covers:
- Separate functions for scoring (
priorityScore()forR-01..R-05) and submission (submitIntake()forR-06andR-07), with each rule ID commented beside the line that implements it. - Explicit tie-breaking in
bankersRound()matching VBScript’s round-half-to-even behavior (U-04) instead of JavaScript’sMath.round(). - An injected
IntakeStoreinterface (insertIntake,insertAudit) and clock parameter onsubmitIntake()so tests never hit a live database, with zero “improvements” to legacy quirks.
The result is small enough to read whole:
REFERENCE CODE src/features/intake/score.ts: priorityScore() and bankersRound() for rules R-01 to R-05 open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
export interface IntakeInput { caseType: string; householdSize: number; daysWaiting: number;}
// VBScript Round() sends a .5 tie to the even neighbour; Math.round() sends it up.export function bankersRound(x: number): number { const floor = Math.floor(x); const diff = x - floor; if (diff < 0.5) return floor; if (diff > 0.5) return floor + 1; return floor % 2 === 0 ? floor : floor + 1;}
export function priorityScore(input: IntakeInput): number { let score = 2; // R-01 if (input.caseType === 'EMERGENCY') score += 4; // R-02 if (input.householdSize >= 5) score += 1; // R-03 score += bankersRound(input.daysWaiting / 20); // R-04 return Math.min(score, 10); // R-05} REFERENCE CODE src/features/intake/submit.ts: submitIntake() and IntakeStore interface for rules R-06 and R-07 open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { priorityScore, type IntakeInput } from './score';
export interface IntakeRow { caseType: string; score: number; status: string; submittedAt: string }export interface AuditRow { caseId: number; action: string; actor: string; at: string }
export interface IntakeStore { insertIntake(row: IntakeRow): Promise<number>; // returns the new case id insertAudit(row: AuditRow): Promise<void>;}
export async function submitIntake( input: IntakeInput, actor: string, store: IntakeStore, clock: () => Date = () => new Date(),) { const caseType = input.caseType.trim().toUpperCase(); // R-02 const score = priorityScore({ ...input, caseType }); const status = score >= 7 ? 'FAST_TRACK' : 'NEW'; // R-06 const caseId = await store.insertIntake({ caseType, score, status, submittedAt: clock().toISOString() }); await store.insertAudit({ caseId, action: 'SUBMIT', actor, at: clock().toISOString() }); // R-07 return { caseId, status, score };}The route handler is thin. The session and role check from lesson 05 runs first in every handler, because the façade is not the authorization boundary:
REFERENCE CODE src/app/api/intake/route.ts: POST handler preserving legacy form fields and response format open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { z } from 'zod';import { submitIntake } from '@/features/intake/submit';import { intakeStore } from '@/features/intake/store';import { withPermission } from '@/lib/auth/with-permission'; // lesson 05: requireSession(store), then the role decision
const IntakeForm = z.object({ caseType: z.string(), householdSize: z.number().int(), daysWaiting: z.number().int(),});
export const POST = withPermission('intake', 'create', async (request, session) => { const form = await request.formData(); const parsed = IntakeForm.safeParse({ caseType: form.get('case_type'), householdSize: Number(form.get('household_size')), daysWaiting: Number(form.get('days_waiting')), }); if (!parsed.success) return new Response('invalid intake', { status: 400 });
const result = await submitIntake(parsed.data, session.sub, intakeStore); return new Response(`${result.caseId}|${result.status}|${result.score}`, { headers: { 'content-type': 'text/plain' }, });});Invalid-input behavior is deliberately outside the corpus: the legacy page raises a runtime error on a non-numeric value (U-05) and nobody has decided whether a 400 is an acceptable replacement. That is an open entry in unknowns.md, not a silent difference.
Step 2: the façade and the rollback
Here are the reference implementations from gov-dev-reference-app for src/features/intake/facade.ts and src/proxy.ts (for comparison with what your agent generates; you do not type them by hand):
export type Target = 'new' | 'legacy';
// One strangled route. Every path not listed here stays on the legacy system.const STRANGLED = new Map<string, string>([['/intake/submit.asp', '/api/intake']]);
export function routeFor(pathname: string, method: string, flag: string | undefined) { const to = STRANGLED.get(pathname); if (to && method === 'POST' && flag === 'new') return { target: 'new' as Target, path: to }; return { target: 'legacy' as Target, path: pathname };}And the reference src/proxy.ts file from gov-dev-reference-app that applies routeFor() to incoming /intake/:path* requests:
import { NextResponse, type NextRequest } from 'next/server';import { routeFor } from '@/features/intake/facade';
export function proxy(request: NextRequest) { const { target, path } = routeFor(request.nextUrl.pathname, request.method, process.env.INTAKE_ROUTE); if (target === 'new') return NextResponse.rewrite(new URL(path, request.url)); return NextResponse.rewrite(new URL(request.nextUrl.pathname + request.nextUrl.search, process.env.LEGACY_ORIGIN));}
export const config = { matcher: ['/intake/:path*'] };The routing decision is a pure function so it can be tested without a server. The setting INTAKE_ROUTE is read per request, which makes rollback a configuration change: set it to anything but new and the same POST goes back to the legacy origin, with no redeploy and no code revert. The default is legacy.
Step 3: the differential harness
Plan the change with your agent
Keep your agent in Plan mode (Shift+Tab, /plan, or the mode picker) and ask it to design the differential test harness in tests/differential/:
We need a differential test harness in tests/differential/ that replays modernization/golden/cases.json through our new intake feature and compares the output and database rows against expected.json. Ask me anything unclear before writing code.What to check in the plan before you approve
- An
Observationcaptures the HTTP response (status,caseId,caseStatus,score), theintakerow written, and theauditrow written, withrunNew(case)runningsubmitIntake()in-process against an in-memory recording store (no live database). diffObservations(legacy, next, tolerances)flattens both observations to dotted paths and reports every differing path not covered by an exact-path tolerance intolerances.ts.tolerances.tsallows only exact dotted paths (no wildcards), a compare function, and a written reason, starting withintake.submittedAt,audit.at, andresponse.caseId.intake.diff.test.tsmaps every mismatch back to the rule IDs inmodernization/rules.md(including fixed field rules likeR-06for status andR-07for audit), prints a rule-coverage table, and fails if any rule inrules.mdhas zero cases.
If the agent’s plan uses wildcard tolerances or ignores the audit row timestamps without validating their format, correct it before approving:
Do not allow wildcards in tolerances.ts. Require exact dotted paths with a written reason and check that both timestamps are well-formed.The harness core:
REFERENCE TEST tests/differential/harness.ts: in-process replay, dotted-path diff, and rule parser open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { submitIntake, type AuditRow, type IntakeRow, type IntakeStore } from '@/features/intake/submit';import type { IntakeInput } from '@/features/intake/score';
export interface Case { id: string; input: IntakeInput; actor: string; exercises: string[]; // rule ids from rules.md that this case probes}
export interface Observation { response: { status: number; caseId: number; caseStatus: string; score: number }; intake: IntakeRow; audit: Omit<AuditRow, 'caseId'>;}
export interface Tolerance { path: string; // exact dotted path, no wildcards reason: string; // why the two systems may legitimately differ compare: (legacy: unknown, next: unknown) => boolean;}
// The new side runs in process against a store that only records. Nothing real is written.export async function runNew(c: Pick<Case, 'input' | 'actor'>): Promise<Observation> { const intake: IntakeRow[] = []; const audit: AuditRow[] = []; const store: IntakeStore = { insertIntake: async (row) => (intake.push(row), intake.length), insertAudit: async (row) => { audit.push(row); }, }; const result = await submitIntake(c.input, c.actor, store); const { caseId: _caseId, ...auditRow } = audit[0]; return { response: { status: 200, caseId: result.caseId, caseStatus: result.status, score: result.score }, intake: intake[0], audit: auditRow, };}
function flatten(v: unknown, prefix = '', out: Record<string, unknown> = {}) { if (v !== null && typeof v === 'object') { for (const [k, x] of Object.entries(v)) flatten(x, prefix ? `${prefix}.${k}` : k, out); } else { out[prefix] = v; } return out;}
// Every field that differs and is not covered by a tolerance.export function diffObservations(legacy: Observation, next: Observation, tolerances: Tolerance[]) { const a = flatten(legacy); const b = flatten(next); const byPath = new Map(tolerances.map((t) => [t.path, t])); const mismatches: { path: string; legacy: unknown; next: unknown }[] = []; for (const path of new Set([...Object.keys(a), ...Object.keys(b)])) { const tol = byPath.get(path); const same = tol ? tol.compare(a[path], b[path]) : Object.is(a[path], b[path]); if (!same) mismatches.push({ path, legacy: a[path], next: b[path] }); } return mismatches;}
// Fields whose rule is fixed by what they are, whatever the case was probing.const FIELD_RULE: Record<string, string> = { 'response.caseStatus': 'R-06', 'intake.status': 'R-06', 'audit.action': 'R-07', 'audit.actor': 'R-07',};
export function rulesFor(path: string, c: Case): string[] { return FIELD_RULE[path] ? [FIELD_RULE[path]] : c.exercises;}
export function parseRules(markdown: string): Map<string, string> { const rules = new Map<string, string>(); for (const line of markdown.split('\n')) { const cells = line.split('|').map((s) => s.trim()); if (/^R-\d+$/.test(cells[1] ?? '')) rules.set(cells[1], cells[2]); } return rules;}The tolerances are the only place the two systems are allowed to differ, and each one carries its reason:
REFERENCE TEST tests/differential/tolerances.ts: exact-path tolerances with written reasons open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import type { Tolerance } from './harness';
// Legacy writes SYSDATETIME() text ("2026-11-04 09:30:00.1234567"); the new code writes ISO-8601 UTC.const looksLikeTimestamp = (v: unknown) => typeof v === 'string' && /^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(\.\d+)?Z?$/.test(v);
const isCaseId = (v: unknown) => Number.isInteger(v) && (v as number) > 0;
export const tolerances: Tolerance[] = [ { path: 'intake.submittedAt', compare: (a, b) => looksLikeTimestamp(a) && looksLikeTimestamp(b), reason: 'wall-clock value and text format differ by design; both must be well-formed timestamps' }, { path: 'audit.at', compare: (a, b) => looksLikeTimestamp(a) && looksLikeTimestamp(b), reason: 'same as intake.submittedAt' }, { path: 'response.caseId', compare: (a, b) => isCaseId(a) && isCaseId(b), reason: 'identity values come from different databases; both must be positive integers' },];And the test that runs the corpus, names the rule, and reports coverage:
REFERENCE TEST tests/differential/intake.diff.test.ts: corpus replay and rule-coverage verification open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { readFileSync } from 'node:fs';import { describe, expect, it } from 'vitest';import { diffObservations, parseRules, rulesFor, runNew, type Case, type Observation } from './harness';import { tolerances } from './tolerances';
const cases: Case[] = JSON.parse(readFileSync('modernization/golden/cases.json', 'utf8'));const golden: Record<string, Observation> = JSON.parse(readFileSync('modernization/golden/expected.json', 'utf8'));const rules = parseRules(readFileSync('modernization/rules.md', 'utf8'));const seen = new Map<string, number>();const failing = new Map<string, number>();
describe('intake: legacy golden corpus vs new feature', () => { for (const c of cases) { it(c.id, async () => { for (const r of c.exercises) seen.set(r, (seen.get(r) ?? 0) + 1); const mismatches = diffObservations(golden[c.id], await runNew(c), tolerances); if (mismatches.length === 0) return;
const named = new Set(mismatches.flatMap((m) => rulesFor(m.path, c))); for (const r of named) failing.set(r, (failing.get(r) ?? 0) + 1); const fields = mismatches.map((m) => ` ${m.path}: legacy=${JSON.stringify(m.legacy)} new=${JSON.stringify(m.next)}`); const why = [...named].map((r) => ` ${r} ${rules.get(r) ?? '(not in rules.md)'}`); throw new Error(`MISMATCH ${c.id}\n${fields.join('\n')}\nrules under test:\n${why.join('\n')}`); }); }
it('reports rule coverage; every rule in rules.md has at least one case', () => { const rows = [...rules.entries()].map(([id, title]) => ({ rule: id, cases: seen.get(id) ?? 0, failing: failing.get(id) ?? 0, title, })); console.table(rows); const uncovered = rows.filter((r) => r.cases === 0).map((r) => r.rule); expect(uncovered, `rules with no golden case: ${uncovered.join(', ')}`).toEqual([]); const strangers = [...seen.keys()].filter((id) => !rules.has(id)); expect(strangers, `cases cite rules missing from rules.md: ${strangers.join(', ')}`).toEqual([]); });});A pass here means two things: every case matched, and every rule in rules.md has a case that ran. Pass/fail alone would let a rule with no case through; the coverage test is why a rule added to rules.md later fails the build until it has a case, or a named test of record (lesson 24’s tested-elsewhere.json).
One guard keeps the tolerance list honest, because the cheapest way to turn a red run green is to widen a tolerance:
REFERENCE TEST tests/differential/tolerances.test.ts: guard against wildcard or business-field tolerances open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { describe, expect, it } from 'vitest';import { tolerances } from './tolerances';
const BUSINESS_FIELDS = [ 'response.score', 'response.caseStatus', 'intake.caseType', 'intake.score', 'intake.status', 'audit.action', 'audit.actor',];
describe('tolerances', () => { it('each has an exact path and a written reason', () => { for (const t of tolerances) { expect(t.path).not.toContain('*'); expect(t.reason.trim().length).toBeGreaterThan(0); } }); it('none covers a business field', () => { const covered = tolerances.map((t) => t.path).filter((p) => BUSINESS_FIELDS.includes(p)); expect(covered).toEqual([]); });});Routing and rollback get their own tests:
REFERENCE TEST tests/differential/facade.test.ts: routing and single-setting rollback tests open reference hide reference Reference implementation from gov-dev-reference-app; open to inspect or compare with what your agent builds.
import { describe, expect, it } from 'vitest';import { routeFor } from '@/features/intake/facade';
describe('intake façade', () => { it('moves the strangled route only when the setting says new', () => { expect(routeFor('/intake/submit.asp', 'POST', 'new')).toEqual({ target: 'new', path: '/api/intake' }); }); it('rolls back with the setting alone: unset, empty or any other value means legacy', () => { for (const flag of [undefined, '', 'legacy', 'NEW', 'off']) { expect(routeFor('/intake/submit.asp', 'POST', flag).target).toBe('legacy'); } }); it('moves nothing else', () => { for (const p of ['/intake/list.asp', '/intake/submit.asp.bak', '/cases/view.asp', '/']) { expect(routeFor(p, 'POST', 'new')).toEqual({ target: 'legacy', path: p }); } }); it('moves only POST; reading the form page stays legacy', () => { expect(routeFor('/intake/submit.asp', 'GET', 'new').target).toBe('legacy'); });});Wire the whole directory as one script, "test:diff": "vitest run tests/differential", and make it a required check on every pull request that touches src/features/intake/ or modernization/. A red run blocks the merge, and the pipeline stage that sets INTAKE_ROUTE=new depends on a green test:diff for the build being promoted. No mismatch is waved through by hand; the only way to make a difference acceptable is a reviewed entry in tolerances.ts, which states why.
Side effects: captured or isolated
The legacy procedure writes two rows. The harness compares them in the same diff as the response, so a missing audit row is a mismatch, not a silent omission. Three rules keep side effects honest:
- Capture what legacy wrote. Lesson 22’s recorder read the
IntakeandIntakeAuditrows the copy wrote and stored them inexpected.json. - Isolate what new writes. The harness injects a recording store, so a test run never touches a real database. When you later mirror live traffic in shadow mode, point the new feature at its own schema or database, never the legacy tables; during a parallel run only the legacy system may write the system of record. Once the route flips, the new feature writes the legacy tables for real, on the legacy clock and actor format, under the rules lesson 24 sets out.
- Stub anything that leaves the building. Mail, queue messages and webhooks sit behind an interface like
IntakeStore, with a recording implementation in tests.
Watch it work
- Typed in the terminal:
claude - Mode:Plan mode
- You type in the session: We need a differential test harness in tests/differential/ that replays modernization/golden/cases.json through our new intake feature and compares the output and database rows against expected.json. Ask me anything unclear before writing code.
- The tool replies: I inspected modernization/golden/cases.json, expected.json, rules.md, and src/features/intake/. I will plan tests/differential/harness.ts, tolerances.ts, and intake.diff.test.ts using an in-memory recording IntakeStore. Which fields should have tolerances initially, and should a mismatch name the R-id from rules.md?
- You type in the session: Allow tolerances only for intake.submittedAt, audit.at, and response.caseId with exact paths and written reasons. Name the rules on mismatch and print a rule-coverage table. Approved.
- The tool replies: Created tests/differential/harness.ts - runNew(): submitIntake() against a recording store, no database - diffObservations(): dotted-path diff, tolerance by exact path - parseRules(): reads the table in modernization/rules.md Created tests/differential/tolerances.ts (3 entries) - intake.submittedAt, audit.at: well-formed timestamps in both formats - response.caseId: positive integer on both sides Created tests/differential/intake.diff.test.ts, tolerances.test.ts, facade.test.ts Added script: test:diff = vitest run tests/differential
- Typed in the terminal:
npm run test:diff - The tool replies: intake: legacy golden corpus vs new feature c01-routine-base ............ pass c02-emergency ............... pass c03-household-five .......... pass c04-wait-half-low ........... pass c05-wait-one-and-half ....... pass c06-wait-two-and-half ....... pass c07-fast-track-at-7 ......... pass c08-fast-track-at-8 ......... pass c09-cap ..................... pass c10-messy-case-type ......... pass reports rule coverage ....... pass rule cases failing title R-01 1 0 Score starts at 2 R-02 2 0 case_type is trimmed and upper-cased; EMERGENCY adds 4 R-03 1 0 household_size >= 5 adds 1 R-04 3 0 adds Round(days_waiting / 20) R-05 1 0 score capped at 10 R-06 3 0 status is FAST_TRACK when score >= 7, otherwise NEW R-07 1 0 every submission writes an audit row with action SUBMIT and the session user Tests: 17 passed (17)
- You type in the session: Simplify bankersRound: just use Math.round(input.daysWaiting / 20) in priorityScore.
- The tool replies: Edited src/features/intake/score.ts - score += bankersRound(input.daysWaiting / 20); + score += Math.round(input.daysWaiting / 20); bankersRound() is now unused. Run the tests before keeping this.
- Typed in the terminal:
npm run test:diff - The tool replies: FAIL c04-wait-half-low Error: MISMATCH c04-wait-half-low response.score: legacy=2 new=3 intake.score: legacy=2 new=3 rules under test: R-04 adds Round(days_waiting / 20) FAIL c06-wait-two-and-half Error: MISMATCH c06-wait-two-and-half response.score: legacy=4 new=5 intake.score: legacy=4 new=5 rules under test: R-04 adds Round(days_waiting / 20) c05-wait-one-and-half passes: 1.5 rounds to 2 under both modes. rule cases failing R-04 3 2 Tests: 2 failed | 15 passed (17)
- You type in the session: Revert that change. Explain why c05 passed while c04 and c06 failed.
- The tool replies: Reverted src/features/intake/score.ts to bankersRound(). c05 (30 days) divides to 1.5. Round-half-to-even and round-half-up both give 2, so the two modes agree there. c04 (10 days, 0.5) gives 0 versus 1, and c06 (50 days, 2.5) gives 2 versus 3, because the even neighbour and the upper neighbour differ. The corpus has one case on the agreeing tie and two on the disagreeing ones; that spread is why the change was caught and is why the tie cases were in the corpus at all. npm run test:diff -> Tests: 17 passed (17)
Illustrative session; exact output varies by tool and agency configuration.
Prove it
Run the clean state first:
npm run test:diffExpected: every case passes and the coverage table lists R-01 to R-07, each with at least one case and zero failing.
Now plant the fault. In src/features/intake/score.ts, replace the R-04 line:
score += bankersRound(input.daysWaiting / 20); // R-04score += Math.round(input.daysWaiting / 20); // R-04Run npm run test:diff again. Expected: the run fails, naming the case, the field, both values and the rule:
Error: MISMATCH c04-wait-half-low response.score: legacy=2 new=3 intake.score: legacy=2 new=3rules under test: R-04 adds Round(days_waiting / 20)The neighbouring cases show the other half: c05 (30 days, 1.5) still passes under both rounding modes, so a corpus with only that tie would have missed the change; c04 and c06 are what catch it. Revert the line and the run is green.
Then try the shortcut and watch the second guard stop it: add a tolerance for response.score to tolerances.ts and tolerances.test.ts fails with response.score in the covered list. The only legitimate way to turn this red run green is to fix the code, or to change rules.md on the business owner’s decision and re-record the corpus.
Two limits to state plainly. The harness calls submitIntake() in process, so route parsing in route.ts and the façade’s rewrite are outside the diff; the façade is covered by its own tests, and the end-to-end path is rehearsed in shadow mode in staging before the setting flips. And the corpus covers ten inputs: it proves the rules you wrote cases for. Cases are the cheapest thing to add when shadow mode shows a divergence, and each divergence found there becomes a permanent case in cases.json, re-recorded from the restored copy.
Mirroring live requests to a staging copy of the new feature moves case data into that environment. Confirm the CUI determination covers it first, as lesson 22 notes, or mirror only the fields the rules read (case type, household size, days waiting) and drop the rest.
The harness fails on every case at audit.at: legacy gives '2026-11-04 09:30:00.1234567' and new gives '2026-11-04T09:30:00.123Z'. What is the right fix?
After a refactor the harness fails on c06: legacy score 4, new score 5. A teammate proposes loosening the response.score comparison to within one point so the pipeline can ship. What happens?
Practice status — replacing one legacy route at a time with evidence of equivalence
| Practice | Status | Also called |
|---|---|---|
| route-by-route replacement behind a façade | common baseline | Strangler Fig; incremental migration |
| golden corpus replayed through old and new | common baseline | characterization tests; golden master; approval tests |
| live traffic mirrored to the new system, legacy answer used | strong optional | shadow mode; parallel run |
| each legitimate difference listed with a path and a reason | strong optional | allowed-diff list; normalizers |
| coverage reported against the written rule list | emerging | requirements traceability |
| rollback as a single routing setting, tested | common baseline | kill switch; traffic switch |
Scale: required | common baseline | strong optional | reference-shop (seen only at organizations that publish their own practice) | emerging
Key takeaway
One route moves behind a façade while the rest stays legacy, and a differential harness replays the golden corpus through both, comparing responses and written rows under field-specific tolerances that each carry a reason. A mismatch fails the run and names the rule from rules.md; a rule with no case fails it too. Rollback is one setting and has its own test. Lesson 24 lets the new feature write the same legacy tables the old app writes, lesson 25 puts agents to work finding the inputs this corpus misses, and the harness keeps guarding every rule through both.
- LEADERSHIP DECISION
- no route moves to the new platform while the differential run is red; a new tolerance is approved by the rule's business owner, with its reason on record
- PRACTITIONER ACTION
- wire test:diff as a required check, flip the routing setting only for a build that passed it, and turn every shadow-mode divergence into a permanent corpus case
- SUCCESS MEASURE
- zero unexplained mismatches at cutover and one-setting rollback verified - contract or revenue risk avoided from decisions changed silently during the port
Legend: LEADERSHIP DECISION = what a leader decides · PRACTITIONER ACTION = what the engineer does next · SUCCESS MEASURE = how you know, in calendar time, assessor-hours or cost, contract or revenue risk, or audit finding avoided