Testing: The Role x Scenario Matrix
Updated
IntermediateWhat you'll learn
~15 min- Build a role-by-scenario matrix that makes authorization coverage visible
- Band test scenarios into four priorities so effort follows risk
- Layer the automation: unit, API, and end-to-end, plus visual regression at the right granularity
The question a test plan must answer
FieldDesk’s rebuild has five roles (field tech, dispatcher, supervisor, admin, read-only auditor) and a few dozen meaningful scenarios. The question a flat list of test cases cannot answer — and the one that matters most after Lesson 4.1 — is: “which role-scenario combinations have we never exercised?” Flat lists hide that hole. A grid exposes it.
The matrix
Switch your workspace agent to Plan mode on your application routes and test suite, and ask it to build the role-by-scenario coverage grid:
Generate a role-by-scenario test matrix for this app across field-tech, dispatcher, supervisor, admin, and auditor. Mark each cell ALLOW, DENY, or N/A, and list every cell (including DENY cells) that currently lacks an automated test.| Scenario | Tech | Dispatcher | Supervisor | Admin | Auditor |
|---|---|---|---|---|---|
| Create request | ALLOW | ALLOW | ALLOW | ALLOW | DENY |
| Assign crew | DENY | ALLOW | ALLOW | ALLOW | DENY |
| Approve overtime work | DENY | DENY | ALLOW | ALLOW | DENY |
| Edit SLA definitions | DENY | DENY | DENY | ALLOW | DENY |
| Export audit log | DENY | DENY | DENY | ALLOW | ALLOW |
The matrix’s power is in the cells people skip: the DENY cells are test cases. “Dispatcher cannot approve overtime” is exactly the horizontal-escalation surface from Lesson 4.1 — and an untested DENY is a control you merely believe exists (the test is four lines: authenticate as dispatcher, call the endpoint, assert 403, assert nothing changed). Empty cells in the coverage report are authorization gaps by construction — that visibility is the whole reason the grid beats the list.
Four priority bands
Not every cell deserves the same rigor on day one. Band the scenarios into a four-tier structure so effort follows risk:
Band 1 Critical journeys - storm-day dispatch, request intake, crew safety flows. Break these, break the utility. Test first, deepest, in every layer.Band 2 Core operations - assignment, SLA timers, notificationsBand 3 Advanced features - reporting, bulk operations, integrationsBand 4 Edge + compliance - a11y sweeps, locale/timezone edges, audit exportBand 4 being last is not band 4 being optional — accessibility and audit-export correctness are procurement and compliance requirements for a utility. The bands sequence the build-out; they never delete it.
Three layers plus one
The matrix says what to prove; the layers say where:
| Layer | Proves | FieldDesk examples |
|---|---|---|
| Unit | Domain logic in isolation | SLA arithmetic, escalation rules |
| API | Contracts + the matrix’s ALLOW/DENY cells | The 403 tests live here — fast, no browser |
| End-to-end | Real journeys through the UI | One golden path per Band-1 scenario |
Keep the pyramid honest: hundreds of unit tests, dozens of API tests, a handful of E2E journeys. E2E is where flake lives; every matrix cell that can be proven at the API layer should be.
Plus one: visual regression, at three granularities — component (does the button render), page (did the dispatch board’s layout survive the CSS change), and flow (screenshot checkpoints inside the Band-1 journeys). Granularity is the knob that controls noise: full-page diffs on every commit drown you in false positives; component-level diffs plus a few page checkpoints catch what matters.
The matrix is generatable (the prompt above), the missing-cell list is mechanical, and the DENY tests are formulaic — ideal AI CLI work, hundreds of cells in an afternoon. The human contribution is the banding judgment and this rule in CI: a new route that adds cells to the matrix without tests for them fails the build. That closes the loop with Lesson 4.1’s ‘no route without a guard’ — now it’s also ‘no guard without a test.’
Coverage tooling shows 84% line coverage and all tests green. The role x scenario matrix shows the auditor column has tests only for its ALLOW cells. What risk is hiding?
Key takeaway
Draw coverage as roles times scenarios; treat DENY cells as first-class tests; band effort by risk; prove cells at the cheapest layer that can prove them, with E2E reserved for golden journeys and visual regression tuned by granularity. The app is built, hardened, and proven — what remains is moving it between environments without breaking either.