Operating in Production: On-Call, Incident Command, and Reporting Clocks Module 5 · Learn

Action Items That Change Something

Last reviewed · content updated

Intermediate

What you'll learn

~16 min
  • Complete a corrective-action register entry with an owner, a due date, a verification criterion, and closure evidence
  • Apply the postmortem-trigger check and the one-register rule to a completed record
  • Replace a per-incident recovery-time scorecard with process measures, without contradicting a service-level recovery metric owned elsewhere

Before the detail — Decision: every corrective action carries an owner, a due date, a verification criterion, and closure evidence, and no record moves to reviewed without them. Outcome: a register that can be audited for whether a fix actually happened, not just for whether a meeting produced a list. Artifact: the corrective-action register, section 10 of the incident record. Status of what follows: common baseline; the reconciliation with a service-level recovery metric below is reusable guidance.

Prompt first: turn a review decision into a closable row

Here is what the review agreed to do about INC-[id] [paste the
decision in one sentence, e.g. "add a saturation signal for the
connection pool"].
Draft a corrective-action register row with these fields, and
nothing else:
OWNER one named person, never a team
DUE a calendar date, not "next sprint"
TYPE prevent | detect | respond
VERIFICATION CRITERION a test someone else could run and get a
yes/no answer - not "confirm it works"
CLOSURE EVIDENCE "open - [what will be attached]" for now
If the decision as I described it has no testable outcome, say so
and ask me for one instead of inventing a vague criterion.

An action with no verification criterion cannot be closed honestly, because nobody agreed in advance what “done” would look like — the prompt refuses to let a well-written sentence paper over that gap.

Four fields, and nothing closes without them

A completed corrective-action register reads like Meridian’s own, closing INC-2026-041: “Make /healthz exercise one pooled database call so the probe cannot stay green through pool exhaustion” — owner named, due date set, verification criterion stated as a test (“with the pool held at capacity in staging, /healthz returns non-200 within 5 s”), and closure evidence a reviewer can actually open (“staging drill log drill-2026-09-10.txt, probe failed in 2.1 s”). A different row in the same register is still open and says exactly what will close it: a CI run link on the pull request that adds a connection-leak test. Closed rows carry evidence someone can open; the open row is honest about what remains.

The federal analog: the improvement plan

HSEEP (the federal exercise-and-evaluation doctrine, not incident-response law) builds its after-action report around an Improvement Plan with tracked corrective actions — the same shape as the register above, evolved for exercises rather than live incidents. 6.1’s exercise doctrine uses the identical register shape for a different reason: a rehearsal finds the gap, not an outage.

One list, checked once

5.1’s postmortem triggers are no longer just a list a reviewer runs by hand — the completed record carries them as a table, section 10’s “Postmortem trigger check”: three columns, Postmortem trigger, Result, Evidence, one row per trigger, each result restricted to yes, no, or missing. INC-2026-041’s table answers yes three times — user-visible downtime, the on-call intervention, and a monitoring failure, because /healthz stayed green through the pool exhaustion that caused it — and closes with “Outcome: REVIEW REQUIRED.” One row answers missing: whether a stakeholder asked for the review on the incident bridge was never written down. Three triggers had already fired, so the missing answer would not have changed the outcome, but a check that quietly writes no when it means nobody knows is a check that will eventually be wrong in the direction that matters, so missing stays missing.

The record’s validator will not accept a reviewed state without this table complete: all six triggers answered, each with evidence, and an Outcome line stated — a record reaching reviewed with a half-run check is rejected, not merely flagged. That check sits inside section 10, beside the corrective-action register it decided was owed, and the postmortem written alongside it keeps exactly one other rule that matters: it points at that register and refuses to keep a second action list of its own, because two lists means one of them is out of date, and it is always the one somebody happens to be reading. The same validator enforces this directly: a postmortem that shows up with its own “Action items” table gets rejected, not warned.

One register instead of two closes off the specific failure an auditor is trained to look for: a corrective action marked done in a narrative document while the register still shows it open.

Process measures, not a duration tile

DevSecOps 6.2 (a separate training in this series) wires a recovery metric from real deployment events, computed per service, and read monthly, as a distribution, by the team that owns the service — an instrument, not a weapon, and it stays exactly that useful as long as nobody lifts it out of that shape. What this lesson rejects is picking the same kind of number up per incident — the scorecard usually labelled MTTR: printing “this incident’s recovery time” next to last month’s number and calling the difference a trend. Google’s own research on the question — Štěpán Davidovič’s “Incident Metrics in SRE” — ran a Monte Carlo simulation over incident data and found that a single incident’s duration, tracked and compared this way, is “poorly suited for decision making or trend analysis.” The two claims do not contradict each other: one is a monthly, distribution-read system signal; the other is a single-incident number invented after the fact to make a report look complete.

In place of a duration tile, this lesson tracks three process measures, each read straight off the record itself: did the reporting clocks in section 6 get met, did the incident-commander handoff recorded in section 4 stay clean, and did every corrective action in section 10 close by its own due date. No verified figure exists anywhere in this domain for what share of an organization’s corrective actions close on time across incidents, and this lesson does not invent one — measure the register’s own due dates against its own closures, never against a benchmark nobody can source.

The three process measures are read directly from the record; a per-incident duration comparison has no statistical footing and can drive a staffing decision on noise — a contract or revenue risk when that decision cuts a rota nobody should have cut.

Stop and escalate when a corrective action’s due date passes with no closure evidence and no revised date recorded in the register — escalate to the action’s owner’s manager and whoever will sign the record’s move to reviewed, because a lapsed action with no explanation is a control that quietly stopped working, and it is exactly the kind of row an assessor samples for.

KNOWLEDGE CHECK

Meridian's operations lead wants a monthly slide comparing this incident's recovery time to last month's, calling it a recovery-time trend. What does this lesson say?

Key takeaway

A corrective action changes something only if it carries an owner, a due date, a verification criterion someone else could run, and closure evidence a reviewer can open — four fields the register enforces and the postmortem never duplicates. HSEEP’s Improvement Plan is the federal shape of the same idea, and 6.1’s exercises use the identical register for a different reason. Where 5.1’s postmortem triggers said a review was owed, this lesson is what the review has to produce to earn the reviewed state. A recovery-time metric is real and useful in DevSecOps 6.2’s shape — per service, monthly, read as a distribution — and a different, unreliable number when lifted per incident and compared to last month’s, which is why this lesson replaces the duration tile with three process measures: clock met, handoff clean, actions closed by due date. Lesson 5.4 reads what five providers and one grid operator wrote when their own registers closed.

LEADERSHIP DECISION require the four fields on every corrective
action, and prohibit per-incident recovery-time
trend reporting in favor of the three process
measures
PRACTITIONER ACTION write owner, due date, verification criterion,
and closure evidence for every agreed action;
keep the postmortem pointed at the register,
never carrying its own list
SUCCESS MEASURE audit finding avoided: zero corrective
actions past due without a revised date or
closure evidence, measured from the register's
own history
Search lessons