Detection as Code and Honest Grading

Detections are software. Manage them like it.

A detection rule is logic that runs in production and pages humans — which makes the click-ops console where most orgs edit them an untested production deployment path with no history, no review, and no rollback. Detection as code applies the boring, proven fix: rules, queries, and response playbooks live in a git repository; changes arrive by pull request with a second engineer’s review; CI validates syntax and schema and runs the rule against replayed sample logs (the true-positive fixture must alert, the benign-noise fixture must not); merges deploy through the pipeline, staged before production. Mainstream platforms now support repository-connected detections natively, and community rule formats compile across SIEMs — the tooling excuse is gone.

What this buys, concretely: the 2 a.m. “who changed the impossible-travel threshold and why” question becomes git log; a bad rule rolls back like any bad deploy; and the fixtures accumulate into a regression suite so next quarter’s edit can’t silently break last quarter’s catch. Meridian already runs this exact loop for infrastructure (the Cloud Modernization training’s policy-gate pattern) — the SOC just joins the same discipline with a different artifact type.

Stand up the detection repo: import our current rules from the console into
source (one file per rule: query, severity, tactic mapping, control tag,
owner, fixtures); build the CI job - lint, schema check, replay tests both
polarities; wire staged deploy dev->prod; and generate the initial fixture
library from last quarter's true positives and top false-positive noise.
Then the starter detections for the Zero Trust streams: denied east-west
attempts (3.3), break-glass use (2.1), activation-ceremony anomalies (2.3),
token-anomaly signals (4.3), quarantine-rate spikes (4.1).

The grading discipline: five values, named owners

Detections tell you what’s happening; alignment grading tells you what you can prove — and it’s where security programs routinely lie to themselves with checkmark tables. The antidote is a scale imported from a real compliance-alignment analysis, whose whole design is making dishonesty structurally awkward:

For each requirement row:   PARTIAL            - works as designed, but the
                                                 designed SCOPE covers only
                                                 part of the requirement
                            PARTIAL-WITH-      - covers the requirement, but
                              DEFECTS            OPERATES defectively - each
                                                 defect observed and listed
                            NONE               - not implemented, say so
                            INHERITED          - satisfied by a platform/
                                                 provider control - name it
                            NOT-APPLICABLE     - with the reason on record
Plus two rules that do the real work:
  1. REFUSE rows you don't own - "this alerting layer is not the collection
     system; retention is the workspace owner's row" - and NAME that owner
  2. CITE evidence for every claim - a file, a query, a witnessed firing

The five values exist because binary compliant/non-compliant erases exactly the information that matters: the difference between “works with known defects” and “doesn’t exist” is the whole risk picture. And the refuse-and-name rule kills the most corrosive habit in compliance mapping — a control layer claiming duties its substrate can’t fulfill. The canonical example from the source analysis: an alerting layer whose only retention lever was a deprecated 365-day cap, graded against a multi-year retention requirement. The honest grade wasn’t a defect of the alerting layer — it was “NONE here; this duty belongs to the storage tier; here’s who owns it” — which converted a fake green checkmark into a real work item for the right team.

Run the grade against the current logging model’s elements quarterly; the output table is simultaneously your gap list, your budget argument, and — Module 6 will show — your accreditation evidence.

AI in the SOC: the honest division of labor

The 2026 state, without vendor gloss: AI triage agents are genuinely good at the volume tier — clustering alerts, enriching entities, drafting investigation summaries, handling the phishing-report queue — and they compress tier-1 toil dramatically. What they don’t do is own anything: investigations still close under an analyst’s name, and detection engineering (what to detect, where thresholds sit, which fixtures define correct) stays a human craft with AI as the drafting hand. The division you already know from the porting and drift lessons holds here: machines gate and triage at volume; humans decide the cases that matter; and the decisions become fixtures so the machines get better. An AI-triaged alert queue with no human closing authority isn’t automation — it’s an unstaffed SOC with good prose.

Key takeaway

Rules in git with fixtures and staged deploys turn the SOC’s logic into reviewable, roll-backable software; the five-value grade with refuse-and-name turns compliance tables into honest work lists; and AI absorbs the triage volume while humans keep the closing authority. What remains for this module is the sharpest verification idea in the training: proving controls by watching them say no.