Test End-to-End with Playwright
Updated
IntermediateWhat you'll learn
~35 min- Sign in for tests without an identity provider by using a test-only session seam and a setup project that saves storageState (a browser's saved cookies, reusable across tests)
- State what a browser suite with no identity provider can prove about sign-in and what only your own provider can prove
- Test role enforcement in the UI and at the API, role-tiered views, and success and failure feedback on a mutation
- Prove the test seam is closed anywhere real users sign in
What a browser test adds, and what it cannot reach
Vitest in Lesson 12 proved that your handlers return the right status for each role. A browser test proves the layer above that: that the page a viewer receives has no control for an action a viewer cannot take, that a failed save is visible to the person who clicked, and that the whole chain from cookie to rendered page holds together.
Sign-in is the awkward part. Signing in against a real identity provider involves redirects you do not control, and commonly multifactor prompts and conditional access (provider policies that decide when a sign-in is allowed). A test account that satisfies those is a standing credential, which many agencies will not issue. So this lesson tests everything on your side of the redirect, and says plainly where the line is.
| Provable without your provider | Needs your identity provider |
|---|---|
Your session layer: cookie set, session expires at SESSION_MAX_SECONDS, a missing or expired cookie is refused | That the provider accepts your app registration and redirect URI |
| Each role sees the controls it can use and no others | That the role names reach your callback as the strings you map in Lesson 5 |
| The API refuses a write the page never offered | That conditional access and multifactor behave as your security team expects |
| Failure and success feedback for a mutation | That the issuer in platform.identity is the provider you mean (a wrong issuer fails every sign-in) |
The right-hand column is a manual check on your first deployment, written down as a checklist item, not skipped.
| Problem | The pattern | Where your platform lesson lands it | You decide |
|---|---|---|---|
| Authenticate in CI with no human | A test-only session seam, guarded by a flag, that mints a session through your own store (the alternative: a dedicated test identity at the agency’s provider, credential held in a vault) | Unchanged on every platform; the vault for a test identity is the platform’s secret store (34, 35, 36) | Whether policy allows a standing test account at all |
| Reuse that sign-in across tests | A setup project saves storageState once per role (the alternative: sign in inside each test) | Unchanged | How many roles you test as |
| Keep the seam out of real use | The route file is not in the repository’s app tree; only the e2e build script copies it in. On top of that: a per-run secret header, a loopback-only test server, and a post-deploy probe (the alternative: a dedicated test identity with no seam at all) | 37: the pipeline step that runs the check and the probe | Which of the layers your pipeline can enforce, and who owns the e2e build script |
| Tests that gate an agent’s work | Protected paths: code owners on the specs, the seam and the checks, plus a hook or diff check | 37: the code host’s owner and required-review settings | Who owns the suite, and who may not |
Plan the change with your agent
Open your workspace agent at the project root, switch to Plan mode (Shift+Tab, /plan, or the desktop app’s mode picker), and ask:
Read src/lib/auth/session.ts and roles.ts, and plan a Playwright end-to-end suite that signs in through a test-only session seam excluded from normal production builds and saves storageState per role. Ask me anything unclear.What to check in the plan before you approve
playwright.config.tsbinds the test server to loopback (next start -H 127.0.0.1), runs a"setup"project before"chromium", and passesE2E_SESSION_SEAM=1and a per-runE2E_SEAM_SECRET.e2e/seam/session.route.tslives outsidesrc/appand returns404unlessE2E_SESSION_SEAM === "1",E2E_SEAM_SECRETis at least 32 characters and matchesx-e2e-seamvia timing-safe comparison,Hostis loopback, and nox-forwarded-forheader is present.scripts/e2e.mjsgenerates a fresh secret per run, copies the seam intosrc/app/api/test/session/route.tsfor the test build, and deletes the copy in afinallyblock, whilescripts/check-no-seam.mjsfails the deploy pipeline if the seam path appears insrc/appor.next/server.e2e/auth.setup.tsseeds"Intake.Viewer","Intake.Operator", and"Intake.Admin", assertsresolveRole()maps each to its closed role, and writese2e/.auth/<role>.json(gitignored alongsidesrc/app/api/test/).
If the agent’s initial plan places the seam permanently under src/app/api/ behind only an environment variable, push back before approving:
Keep the seam route outside src/app so normal builds exclude it, and require a per-run 32-character secret header on loopback.- Typed in the terminal:
claude - Mode:Plan mode
- You type in the session: Read src/lib/auth/session.ts and roles.ts, and plan a Playwright end-to-end suite that signs in through a test-only session seam excluded from normal production builds and saves storageState per role. Ask me anything unclear.
- The tool replies: Proposed plan: 1. e2e/seam/session.route.ts: test session endpoint stored outside src/app, minting sessions in the app's SessionStore with SESSION_COOKIE. 2. scripts/e2e.mjs and scripts/check-no-seam.mjs: copy the seam in only during an e2e build with a per-run secret, remove it in finally, and verify deploy builds never contain /api/test/session. 3. playwright.config.ts and e2e/auth.setup.ts: loopback server, setup project seeding Intake.Viewer, Intake.Operator, and Intake.Admin into e2e/.auth/<role>.json. Should the seam also verify a per-run secret header and loopback Host in case a test build is ever deployed by mistake?
- You type in the session: Keep the seam route outside src/app so normal builds exclude it, and require a per-run 32-character secret header on loopback. Build it.
- The tool replies: Created: - playwright.config.ts: setup project plus chromium depending on it, webServer bound to 127.0.0.1 - e2e/seam/session.route.ts: the seam, outside src/app - scripts/e2e.mjs: per-run secret, copy the route in, build, test, delete the copy - scripts/check-no-seam.mjs: deploy-pipeline check - e2e/seam.ts and e2e/auth.setup.ts: three roles seeded with raw provider role names (Intake.Viewer, Intake.Operator, Intake.Admin), a check that resolveRole() maps each to a role, storageState per role - .gitignore entries for e2e/.auth/ and src/app/api/test/ Two things I could not confirm from here: 1. The cookie is Secure and the test server is plain http on loopback. Chromium accepts Secure cookies on localhost, but I have not confirmed that Playwright's request-context cookie jar keeps one over http. If setup fails to authenticate, serve the test app over https with a local certificate. 2. I am relying on Next only discovering routes under src/app. I believe that holds, but I have not run it against your pinned major. check-no-seam.mjs and the probe are how you confirm it on a real build.
- You type in the session: The second point worries me. How do I make sure it never opens in a real environment?
- The tool replies: A flag alone is not enough: if it is ever left on in a deployed environment, anyone can mint an admin session. So the flag is the last of four layers, not the only one. 1. The route is not in the deployable build. It lives in e2e/seam and only scripts/e2e.mjs copies it into src/app for the test build, then deletes it. A normal build has no such route to open. 2. If a test build is ever deployed anyway, the route still needs the flag, a 32-character secret generated for that one run, a matching header, and a loopback Host with no forwarded-for header. The test server itself binds to 127.0.0.1, which is the check that does not trust a header. 3. check-no-seam.mjs fails the deploy pipeline if the route file or the string /api/test/session is in the build output. 4. A probe test posts to the seam on the deployed URL and expects 404. It is the detector for when 1 to 3 have all failed. None of this makes the seam safe to ship. It makes shipping it a sequence of independent mistakes. Do you want me to draft the probe?
- You type in the session: Yes, and add the viewer, operator and admin specs for the vehicle-fleet feature.
- The tool replies: I will add e2e/seam-closed.spec.ts (skipped unless PROBE_URL is set), e2e/roles.spec.ts, e2e/tiers.spec.ts and e2e/mutation-feedback.spec.ts. The page and button labels I use come from the vehicle-fleet page; where I guessed a label I have marked it with a TODO so you can check it against the real UI.
- Typed in the terminal:
node scripts/e2e.mjs, then tell me which of these files I should not let you edit. - The tool replies: The viewer spec failed on the first run: the Add vehicle button was visible to the viewer. That is the suite doing its job, and the fix belongs in the page's visibility condition, which I changed. I did not change the spec. On your question: you should not let me edit the files that decide whether my change passes. That is e2e/roles.spec.ts, e2e/tiers.spec.ts, e2e/mutation-feedback.spec.ts, e2e/seam-closed.spec.ts, e2e/seam/, scripts/e2e.mjs, scripts/check-no-seam.mjs, playwright.config.ts and CODEOWNERS itself. A line in your prompt telling me to leave them alone is a request. The control is code owners with required review on those paths, plus a check that fails when a diff touches them. Lesson 25 builds that check.
Illustrative session; exact output varies by tool and agency configuration.
The seam and the setup project
Read the seam first, because it is the riskiest file in the suite: with it, anyone can become any role. A single runtime flag is not a control strong enough for that. If the flag is ever set in a deployed environment, the route is live. So the first control is that the route is not in the deployable build at all, and the flag, secret and loopback check are what remain if that fails.
Layer 1: keep the route out of deployable builds. Next discovers App Router routes by walking src/app. The seam therefore lives in e2e/seam/, outside that tree, and scripts/e2e.mjs copies it into place for the test build only:
REFERENCE CODE scripts/e2e.mjs: test-only build wrapper that injects and cleans up the session seam open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
// scripts/e2e.mjs: the only code path that puts the seam into a buildimport { randomBytes } from "node:crypto";import { cpSync, mkdirSync, rmSync } from "node:fs";import { spawnSync } from "node:child_process";
const target = "src/app/api/test/session";const secret = randomBytes(32).toString("hex"); // new for every run, never written to diskconst env = { ...process.env, E2E_SESSION_SEAM: "1", E2E_SEAM_SECRET: secret };let status = 1;try { mkdirSync(target, { recursive: true }); cpSync("e2e/seam/session.route.ts", `${target}/route.ts`); status = spawnSync("npx", ["next", "build"], { stdio: "inherit", env }).status ?? 1; if (status === 0) status = spawnSync("npx", ["playwright", "test", ...process.argv.slice(2)], { stdio: "inherit", env }).status ?? 1;} finally { rmSync("src/app/api/test", { recursive: true, force: true }); // the copy never outlives the run}process.exit(status); REFERENCE CODE scripts/check-no-seam.mjs: post-build check verifying the seam is absent from deployable builds open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
// scripts/check-no-seam.mjs: run in the deploy pipeline after the normal buildimport { existsSync, readdirSync, readFileSync } from "node:fs";import { join } from "node:path";
function* walk(dir) { for (const e of readdirSync(dir, { withFileTypes: true })) { const p = join(dir, e.name); if (e.isDirectory()) yield* walk(p); else yield p; }}const problems = [];if (existsSync("src/app/api/test")) problems.push("src/app/api/test exists");if (!existsSync(".next/server")) problems.push("no .next/server: run this after the build");else for (const f of walk(".next/server")) { if (readFileSync(f, "latin1").includes("/api/test/session")) problems.push(`seam path found in ${f}`);}if (problems.length) { console.error("check-no-seam:", problems.join("; ")); process.exit(1); }console.log("check-no-seam: ok");That Next serves only routes under src/app is the documented App Router behavior, and this lesson relies on it; it was not re-run against a build for this lesson. check-no-seam.mjs is how you confirm it on your own build: plant the file in src/app/api/test/session/route.ts, build, and the check must fail; remove it, rebuild, and the check must pass. A pageExtensions setting that only a test build enables is another way to exclude a file, and it is a pattern to verify the same way: the setting also decides which files count as route files, so confirm what it does to route.ts in your version before you rely on it.
Layers 2 and 3: a per-run secret and loopback only. If a test build is ever deployed, the route still has to refuse everyone who does not hold this run’s secret:
REFERENCE CODE e2e/seam/session.route.ts: test session seam gated by per-run secret and loopback checks open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
// e2e/seam/session.route.ts (copied to src/app/api/test/session/route.ts by scripts/e2e.mjs only)import { createHash, timingSafeEqual } from "node:crypto";import { cookies } from "next/headers";import { NextRequest, NextResponse } from "next/server";import { z } from "zod";import { newSessionId } from "@/lib/auth/session";import { SESSION_COOKIE } from "@/lib/auth/session-config";import { store } from "@/lib/auth/store";
const Body = z.object({ roles: z.array(z.string()).max(8), ageSeconds: z.number().int().min(0).optional(),});
const refuse = () => new NextResponse(null, { status: 404 }); // one answer for every refusalconst digest = (v: string) => createHash("sha256").update(v).digest();
function allowed(request: NextRequest): boolean { const secret = process.env.E2E_SEAM_SECRET ?? ""; if (process.env.E2E_SESSION_SEAM !== "1" || secret.length < 32) return false; const given = request.headers.get("x-e2e-seam") ?? ""; if (!timingSafeEqual(digest(given), digest(secret))) return false; // Advisory only: both headers are client-controlled. The real loopback control is that the // test server is started with `next start -H 127.0.0.1`, so nothing off-host can connect. const host = (request.headers.get("host") ?? "").split(":")[0]; if (host !== "127.0.0.1" && host !== "localhost") return false; return !request.headers.has("x-forwarded-for");}
export async function POST(request: NextRequest) { if (!allowed(request)) return refuse();
const parsed = Body.safeParse(await request.json().catch(() => null)); if (!parsed.success) return new NextResponse(null, { status: 400 });
const id = newSessionId(); await store.set(id, { createdAt: Date.now() - (parsed.data.ageSeconds ?? 0) * 1000, user: { sub: "e2e-user", issuer: "https://idp.e2e.test", roles: parsed.data.roles, name: "E2E User" }, }); (await cookies()).set(SESSION_COOKIE, id, { httpOnly: true, secure: true, sameSite: "lax", path: "/" }); return new NextResponse(null, { status: 204 });}Here is the reference header helper in e2e/seam.ts that your agent generates for test calls to the seam:
export function seamHeaders(): Record<string, string> { const secret = process.env.E2E_SEAM_SECRET; if (!secret) throw new Error("E2E_SEAM_SECRET is not set: run the suite through scripts/e2e.mjs"); return { "x-e2e-seam": secret };} REFERENCE TEST e2e/auth.setup.ts: Playwright setup project seeding storageState per role open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { expect, test as setup } from "@playwright/test";import { seamHeaders } from "./seam";
import { resolveRole, type Role } from "../src/lib/auth/roles";
// Raw role names, as a provider sends them. Lesson 5's ROLE_CLAIM_MAP keys on these exact// strings; a short name such as "operator" would resolve to null and every request would be denied.const SEEDS: Record<"viewer" | "operator" | "admin", { names: string[]; role: Role }> = { viewer: { names: ["Intake.Viewer"], role: "viewer" }, operator: { names: ["Intake.Operator"], role: "operator" }, admin: { names: ["Intake.Admin"], role: "admin" },};
for (const [name, seed] of Object.entries(SEEDS)) { setup(`seed a ${name} session`, async ({ request }) => { // The seam stores whatever it is given, so check here that the app will accept it. expect(resolveRole(seed.names), `${seed.names.join(",")} does not resolve to a role`).not.toBeNull(); expect(resolveRole(seed.names)).toBe(seed.role); const res = await request.post("/api/test/session", { data: { roles: seed.names }, headers: seamHeaders() }); expect(res.status()).toBe(204); await request.storageState({ path: `e2e/.auth/${name}.json` }); });} REFERENCE CONFIG playwright.config.ts: loopback test server and setup project dependencies open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { defineConfig, devices } from "@playwright/test";
const baseURL = process.env.E2E_BASE_URL ?? "http://127.0.0.1:3000";
export default defineConfig({ testDir: "./e2e", fullyParallel: true, forbidOnly: !!process.env.CI, retries: process.env.CI ? 1 : 0, reporter: process.env.CI ? [["junit", { outputFile: "results/e2e.xml" }], ["html", { open: "never" }]] : [["html", { open: "never" }]], use: { baseURL, colorScheme: "dark", trace: "on-first-retry" }, webServer: { // scripts/e2e.mjs has already built. Bound to loopback so nothing off-host can reach the seam. command: "npx next start -H 127.0.0.1 -p 3000", url: baseURL, reuseExistingServer: false, // never attach to a server you did not start with this run's secret timeout: 180_000, env: { E2E_SESSION_SEAM: "1", E2E_SEAM_SECRET: process.env.E2E_SEAM_SECRET ?? "" }, }, projects: [ { name: "setup", testMatch: /.*\.setup\.ts/ }, { name: "chromium", use: { ...devices["Desktop Chrome"] }, testIgnore: /.*\.setup\.ts/, dependencies: ["setup"], }, ],});Run the suite with node scripts/e2e.mjs, not npx playwright test: the wrapper is what creates the secret and builds with the seam. Confirm next start -H against npx next start --help for your pinned major.
Residual risk. These layers make a mistake harder; they do not make the seam safe to ship. A test build that is deployed anyway still contains a route that mints sessions, and what stands between it and an attacker is a 256-bit secret that exists only in that run’s environment. That secret can leak through CI logs or a debug dump of the environment, and the loopback header checks are advisory because a client can send any header. The copy step relies on a script and a .gitignore line that a person can bypass by committing the file into src/app. Layer 4, the deployed-environment probe below, is the detector for all of these, and it only runs after a deploy. Give the seam a test-only session store, never the real one, so that even a leaked seam cannot write into a store real users sign in through.
The seam stores the roles it is given as raw role names, the strings Lesson 5’s ROLE_CLAIM_MAP recognizes, and resolveRole() ignores anything else. A seam that stored viewer would mint a session that every request denies, so the setup project asserts that each seeded name resolves to a role before it saves storageState.
Apart from its gates, the seam creates the session through the same store and cookie flags as the real callback, so every test after it exercises your real session code. What it replaces is only the part you cannot automate: the identity provider proving who the person is. The stored user has the same shape the real callback stores, { sub, issuer, roles, name }, so requireSession() cannot tell the difference.
The saved files hold session cookies. They expire with the session, but a committed cookie file is a habit worth never starting. The setup project creates them fresh on every run.
Role enforcement: the page and the API
Test both layers. The page test shows what a role sees; the API test shows that hiding a button was never the control.
REFERENCE TEST e2e/roles.spec.ts: browser and API role-enforcement specs open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { expect, test } from "@playwright/test";
test.describe("viewer", () => { test.use({ storageState: "e2e/.auth/viewer.json" });
test("sees the list and no write controls", async ({ page }) => { await page.goto("/vehicle-fleet"); await expect(page.getByRole("grid")).toBeVisible(); await expect(page.getByRole("button", { name: "Add vehicle" })).toHaveCount(0); await expect(page.getByRole("button", { name: /^Delete/ })).toHaveCount(0); });
test("the API refuses a write the page never offered", async ({ page }) => { const res = await page.request.post("/api/vehicle-fleet", { data: { vehicle_id: "FL-1001", make: "Ford", model: "F-150", year: 2024 }, }); expect(res.status()).toBe(403); });});
test.describe("operator", () => { test.use({ storageState: "e2e/.auth/operator.json" });
test("can add a vehicle but not delete one", async ({ page }) => { await page.goto("/vehicle-fleet"); await expect(page.getByRole("button", { name: "Add vehicle" })).toBeVisible(); await expect(page.getByRole("button", { name: /^Delete/ })).toHaveCount(0); });});
test.describe("no session", () => { test.use({ storageState: { cookies: [], origins: [] } });
test("the page redirects to sign-in without following it", async ({ request }) => { const res = await request.get("/vehicle-fleet", { maxRedirects: 0 }); expect(res.status()).toBe(307); expect(res.headers()["location"]).toContain("/auth/login"); });
test("the API answers 401 with the standard error shape", async ({ request }) => { const res = await request.get("/api/vehicle-fleet"); expect(res.status()).toBe(401); expect((await res.json()).error).toBeDefined(); });});The no-session test stops at the first redirect on purpose. Following it would send the browser to the real sign-in host, which is exactly the part this suite cannot test.
An expired session is the third case worth a test. The seam takes ageSeconds, so your agent writes this reference spec in e2e/session-expiry.spec.ts to mint a session that is already past the lifetime:
import { expect, test } from "@playwright/test";import { SESSION_MAX_SECONDS } from "../src/lib/auth/session-config";import { seamHeaders } from "./seam";
test("a session older than SESSION_MAX_SECONDS is refused", async ({ request }) => { await request.post("/api/test/session", { data: { roles: ["Intake.Admin"], ageSeconds: SESSION_MAX_SECONDS + 1 }, headers: seamHeaders(), }); const res = await request.get("/api/vehicle-fleet"); expect(res.status()).toBe(401);});Role-tiered views
Lesson 11’s pattern has one contract: no tier shows a control the role cannot invoke, and the tiers a user is not looking at are hidden from assistive technology. Both can be asserted by walking every tier as every role.
REFERENCE TEST e2e/tiers.spec.ts: role-by-tier visibility and single-tabpanel specs open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { expect, test } from "@playwright/test";
for (const role of ["viewer", "operator"] as const) { test.describe(`${role} tiers`, () => { test.use({ storageState: `e2e/.auth/${role}.json` });
test("every tier hides the controls this role cannot use", async ({ page }) => { await page.goto("/vehicle-fleet"); for (const tab of await page.getByRole("tab").all()) { await tab.click(); await expect(page.getByRole("tabpanel")).toHaveCount(1); // the other tiers are not in the accessibility tree if (role === "viewer") { await expect(page.getByRole("button", { name: /Add vehicle|Delete/ })).toHaveCount(0); } } }); });}getByRole ignores elements hidden from assistive technology, which is why a count of one tab panel is a real assertion about what a screen reader would find.
Mutation feedback
Silent success and silent failure both generate the support ticket that arrives two weeks later. Test the two states a person sees. This test intercepts the network so it can force each outcome, which means it proves the page handles the contract from Lesson 7, not that your database works.
REFERENCE TEST e2e/mutation-feedback.spec.ts: visible success and failure feedback specs open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { expect, test } from "@playwright/test";
test.use({ storageState: "e2e/.auth/operator.json" });
async function fillAndSubmit(page: import("@playwright/test").Page) { await page.getByRole("button", { name: "Add vehicle" }).click(); await page.getByLabel("Vehicle ID").fill("FL-1001"); await page.getByLabel("Make").fill("Ford"); await page.getByLabel("Model").fill("F-150"); await page.getByLabel("Year").fill("2024"); await page.getByRole("button", { name: "Save" }).click();}
test("success is announced and the dialog closes", async ({ page }) => { await page.route("**/api/vehicle-fleet", async (route) => { if (route.request().method() !== "POST") return route.fallback(); await route.fulfill({ status: 201, json: { data: { id: 99, vehicle_id: "FL-1001" } } }); }); await page.goto("/vehicle-fleet"); await fillAndSubmit(page); await expect(page.getByRole("alert")).toContainText("added"); await expect(page.getByRole("dialog")).toHaveCount(0);});
test("failure shows the safe message, keeps the form, and shows no internals", async ({ page }) => { await page.route("**/api/vehicle-fleet", async (route) => { if (route.request().method() !== "POST") return route.fallback(); await route.fulfill({ status: 500, json: { error: { code: "INTERNAL_ERROR", message: "Something went wrong. Try again or contact support." } }, }); }); await page.goto("/vehicle-fleet"); await fillAndSubmit(page); await expect(page.getByRole("alert")).toContainText("Something went wrong"); await expect(page.getByRole("dialog")).toBeVisible(); await expect(page.locator("body")).not.toContainText(/INTERNAL_ERROR|stack|mssql/i);});If the button labels above do not match your page, fix the labels, not the assertions. A role-and-name locator also doubles as a check that the control has an accessible name, which Lesson 14 relies on.
The route test in Lesson 12 proves the server produces the error shape. This test proves the page renders that shape. The seam between them is the response contract, so share its type between both and a change on one side breaks the other in CI.
These specs gate your own work, so they are not yours to edit
The failure in the session above is the situation this pattern exists for. A red viewer spec can be fixed in the page or by deleting the assertion, and only one of those is a fix. For the same reason the seam, the build wrapper and the probe sit on the protected list: an agent that can edit check-no-seam.mjs can make the build pass with the seam still in it.
This is Lesson 25’s referee pattern, applied here: the agent proposes, a protected test decides. Put the specs, e2e/seam/, scripts/e2e.mjs, scripts/check-no-seam.mjs, playwright.config.ts and CODEOWNERS under named code owners with required review (the same setup Lesson 12 shows for the Vitest files), and add a hook or diff check that fails when a change touches them. Lesson 25 builds that checker: every protected path must have an owner, and the list cannot shrink. A prompt instruction is not a substitute for either.
Prove it
Run the suite, then prove the seam is out of the deployable build and closed in the deployed environment.
node scripts/e2e.mjsnpm run build && node scripts/check-no-seam.mjsThe second command is the deploy-pipeline check: a normal build, then a scan for the seam. The probe is the fourth layer. It runs against any URL you give it and passes only if the seam refuses everyone there, with no secret, with a made-up secret, and with a forged loopback host:
REFERENCE TEST e2e/seam-closed.spec.ts: post-deploy probe verifying the session seam returns 404 open reference hide reference Reference pattern for your workspace agent; open to inspect or compare this file. You do not type or paste this by hand.
import { expect, test } from "@playwright/test";
test.skip(!process.env.PROBE_URL, "Set PROBE_URL to the environment to probe");
test("the test session seam is closed", async ({ playwright }) => { const ctx = await playwright.request.newContext({ baseURL: process.env.PROBE_URL }); const attempts: Record<string, string>[] = [ {}, { "x-e2e-seam": "0".repeat(64) }, { "x-e2e-seam": "0".repeat(64), host: "127.0.0.1" }, ]; for (const headers of attempts) { const res = await ctx.post("/api/test/session", { data: { roles: ["Intake.Admin"] }, headers }); expect(res.status(), JSON.stringify(headers)).toBe(404); }});Run the probe against your deployed URL:
PROBE_URL=https://staging.example.gov npx playwright test e2e/seam-closed.spec.ts --project=chromiumPlanted faults, each of which must turn a named test red:
| Plant this fault | This test must fail |
|---|---|
| Remove the role check from the Add vehicle button’s visibility condition | viewer: sees the list and no write controls |
Remove withPermission() from the POST handler | viewer: the API refuses a write the page never offered |
Set E2E_SESSION_SEAM=1 on the staging app | no test fails: the route is absent, so this fault alone changes nothing. That is layer 1 working. |
Copy e2e/seam/session.route.ts into src/app/api/test/session/ and build normally | check-no-seam.mjs exits non-zero |
| Deploy a test build to staging with the flag and a secret set, then probe it | the test session seam is closed (the probe sends no valid secret) |
| Remove the secret comparison from the route, deploy the test build | the test session seam is closed |
| Make the Save handler swallow errors | failure shows the safe message |
What the suite does not prove is on the right-hand side of the first table. Write those items on your first-deployment checklist today.
A developer sets the seam flag in the staging app settings to debug a failure. Which control means that flag alone does not open an admin-minting route?
Which statement can a suite that uses the session seam honestly make?
Key takeaway
Replace only the step you cannot automate. A guarded test seam mints sessions through your real store, a setup project saves one storageState per role, and the specs then test what is yours: role enforcement at the page and the API, role-tiered views, expiry, and visible failure. Keep the seam out of the deployable build, gate it with a per-run secret and loopback, and keep a probe that proves it is closed wherever real people sign in, and keep a written list of what only your identity provider can confirm.
- LEADERSHIP DECISION
- Whether the test seam may exist in any build, and who signs the list of what only the identity provider can confirm.
- PRACTITIONER ACTION
- Run the per-role specs, then run the probe that proves the seam is closed in the deployable build.
- SUCCESS MEASURE
- Audit finding avoided: no session-minting route is reachable where real people sign in.
Legend: LEADERSHIP DECISION = what a leader decides · PRACTITIONER ACTION = what the engineer does next · SUCCESS MEASURE = how you know, in calendar time, assessor-hours or cost, contract or revenue risk, or audit finding avoided