Testing: The Role x Scenario Matrix

The question a test plan must answer

FieldDesk’s rebuild has five roles (field tech, dispatcher, supervisor, admin, read-only auditor) and a few dozen meaningful scenarios. The question a flat list of test cases cannot answer — and the one that matters most after Lesson 4.1 — is: “which role-scenario combinations have we never exercised?” Flat lists hide that hole. A grid exposes it.

The matrix

Generate a role x scenario test matrix for this app. Rows: every user-facing
scenario, grouped by workflow (intake, dispatch, SLA, reporting, admin).
Columns: field-tech, dispatcher, supervisor, admin, auditor. Each cell:
ALLOW (expected to work), DENY (expected to be blocked), or N/A (scenario
meaningless for that role). Then list every cell we have no test for --
including the DENY cells.

Scenario	Tech	Dispatcher	Supervisor	Admin	Auditor
Create request	ALLOW	ALLOW	ALLOW	ALLOW	DENY
Assign crew	DENY	ALLOW	ALLOW	ALLOW	DENY
Approve overtime work	DENY	DENY	ALLOW	ALLOW	DENY
Edit SLA definitions	DENY	DENY	DENY	ALLOW	DENY
Export audit log	DENY	DENY	DENY	ALLOW	ALLOW

The matrix’s power is in the cells people skip: the DENY cells are test cases. “Dispatcher cannot approve overtime” is exactly the horizontal-escalation surface from Lesson 4.1 — and an untested DENY is a control you merely believe exists (the test is four lines: authenticate as dispatcher, call the endpoint, assert 403, assert nothing changed). Empty cells in the coverage report are authorization gaps by construction — that visibility is the whole reason the grid beats the list.

Four priority bands

Not every cell deserves the same rigor on day one. Band the scenarios so effort follows risk:

Band 1  Critical journeys   - storm-day dispatch, request intake, crew safety
                              flows. Break these, break the utility. Test first,
                              deepest, in every layer.
Band 2  Core operations     - assignment, SLA timers, notifications
Band 3  Advanced features   - reporting, bulk operations, integrations
Band 4  Edge + compliance   - a11y sweeps, locale/timezone edges, audit export

Band 4 being last is not band 4 being optional — accessibility and audit-export correctness are procurement and compliance requirements for a utility. The bands sequence the build-out; they never delete it.

Three layers plus one

The matrix says what to prove; the layers say where:

Layer	Proves	FieldDesk examples
Unit	Domain logic in isolation	SLA arithmetic, escalation rules
API	Contracts + the matrix’s ALLOW/DENY cells	The 403 tests live here — fast, no browser
End-to-end	Real journeys through the UI	One golden path per Band-1 scenario

Keep the pyramid honest: hundreds of unit tests, dozens of API tests, a handful of E2E journeys. E2E is where flake lives; every matrix cell that can be proven at the API layer should be.

Plus one: visual regression, at three granularities — component (does the button render), page (did the dispatch board’s layout survive the CSS change), and flow (screenshot checkpoints inside the Band-1 journeys). Granularity is the knob that controls noise: full-page diffs on every commit drown you in false positives; component-level diffs plus a few page checkpoints catch what matters.

💡Let the AI fill the grid, let CI guard it

The matrix is generatable (the prompt above), the missing-cell list is mechanical, and the DENY tests are formulaic — ideal AI CLI work, hundreds of cells in an afternoon. The human contribution is the banding judgment and this rule in CI: a new route that adds cells to the matrix without tests for them fails the build. That closes the loop with Lesson 4.1’s ‘no route without a guard’ — now it’s also ‘no guard without a test.‘

KNOWLEDGE CHECK

Coverage tooling shows 84% line coverage and all tests green. The role x scenario matrix shows the auditor column has tests only for its ALLOW cells. What risk is hiding?

Key takeaway

Draw coverage as roles times scenarios; treat DENY cells as first-class tests; band effort by risk; prove cells at the cheapest layer that can prove them, with E2E reserved for golden journeys and visual regression tuned by granularity. The app is built, hardened, and proven — what remains is moving it between environments without breaking either.