Detection as Code and Honest Grading
Updated
IntermediateWhat you'll learn
~18 min- Manage detections like software: git, review, CI validation, staged deploy
- Grade control alignment on the five-value scale - and refuse to claim rows you don't own
- Use AI triage where it's strong while keeping detection engineering human-owned
Detections are software. Manage them like it.
A detection rule is logic that runs in production and pages humans — which makes the click-ops console where most orgs edit them an untested production deployment path with no history, no review, and no rollback. Detection as code applies the boring, proven fix: rules, queries, and response playbooks live in a git repository; changes arrive by pull request with a second engineer’s review; CI validates syntax and schema and runs the rule against replayed sample logs (the true-positive fixture must alert, the benign-noise fixture must not); merges deploy through the pipeline, staged before production. Mainstream platforms now support repository-connected detections natively, and community rule formats compile across SIEMs — the tooling excuse is gone.
What this buys, concretely: the 2 a.m. “who changed the impossible-travel threshold and why” question becomes git log; a bad rule rolls back like any bad deploy; and the fixtures accumulate into a regression suite so next quarter’s edit can’t silently break last quarter’s catch. Meridian already runs this exact loop for infrastructure (the Cloud Modernization training’s policy-gate pattern) — the SOC just joins the same discipline with a different artifact type.
Prompt first
Switch your workspace agent into plan mode on your exported console rules and log samples, then start with (if you do not have live exports or files handy, ask it to generate a small Meridian practice sample from the lesson’s scenario first):
Plan a detection-as-code repository with CI log-replay fixtures, stageddeployment, and starter detections for our Zero Trust control streams. Askclarifying questions about our SIEM rule format and log schemas first.Check that the plan or draft covers:
- Repository layout importing existing console rules into version control (one file per rule containing query, severity, tactic mapping, control tag, owner, and replay fixtures)
- CI pipeline running syntax lint, schema validation, and log-replay tests in both polarities (true-positive fixtures must alert; benign-noise fixtures must not) before staged
dev -> proddeployment - Starter Zero Trust detection rules covering denied east-west attempts (Lesson 3.3), break-glass account usage (Lesson 2.1), activation-ceremony anomalies (Lesson 2.3), token anomalies (Lesson 4.3), and washroom quarantine spikes (Lesson 4.1)
If any rule lacks a negative noise fixture or control-ID metadata, follow up with:
Require both a true-positive and a benign-noise log fixture plus a namedowner team and control tag before CI allows any detection rule to merge.The grading discipline: five values, named owners
Detections tell you what’s happening; alignment grading tells you what you can prove, and it’s where security programs routinely lie to themselves with checkmark tables. The antidote is a scale imported from a real compliance-alignment analysis, whose whole design is making dishonesty structurally awkward. The five-value grading structure looks like:
For each requirement row: PARTIAL - works as designed, but the designed SCOPE covers only part of the requirement PARTIAL-WITH- - covers the requirement, but DEFECTS OPERATES defectively - each defect observed and listed NONE - not implemented, say so INHERITED - a platform/provider control supports evidence for the row - name it NOT-APPLICABLE - with the reason on recordPlus two rules that do the real work: 1. REFUSE rows you don't own - "this alerting layer is not the collection system; retention is the workspace owner's row" - and NAME that owner 2. CITE evidence for every claim - a file, a query, a witnessed firingThe five values exist because binary compliant/non-compliant erases exactly the information that matters: the difference between “works with known defects” and “doesn’t exist” is the whole risk picture. And the refuse-and-name rule kills the most corrosive habit in compliance mapping — a control layer claiming duties its substrate can’t fulfill. The canonical example from the source analysis: an alerting layer whose only retention lever was a deprecated 365-day cap, graded against a multi-year retention requirement. The honest grade wasn’t a defect of the alerting layer — it was “NONE here; this duty belongs to the storage tier; here’s who owns it” — which converted a fake green checkmark into a real work item for the right team.
Run the grade against the current logging model’s elements quarterly; the output table is simultaneously your gap list, your budget argument, and — Module 6 will show — your accreditation evidence.
AI in the SOC: the honest division of labor
The 2026 state, without vendor gloss: AI triage agents are genuinely good at the volume tier — clustering alerts, enriching entities, drafting investigation summaries, handling the phishing-report queue — and they compress tier-1 toil substantially. What they don’t do is own anything: investigations still close under an analyst’s name, and detection engineering (what to detect, where thresholds sit, which fixtures define correct) stays a human craft with AI as the drafting hand. The division you already know from the porting and drift lessons holds here: machines gate and triage at volume; humans decide the cases that matter; and the decisions become fixtures so the machines get better. An AI-triaged alert queue with no human closing authority isn’t automation — it’s an unstaffed SOC with good prose.
An auditor asks Meridian to demonstrate compliance with a log-integrity requirement ('detect and alert when logging stops'). The team's old mapping table shows a green checkmark citing the alert library. Under this lesson's grading, what should the row actually say — given the canary exists for the central workspace, but a third of systems still log to scattered workspaces with no canary?
Key takeaway
Rules in git with fixtures and staged deploys turn the SOC’s logic into reviewable, roll-backable software; the five-value grade with refuse-and-name turns compliance tables into honest work lists; and AI absorbs the triage volume while humans keep the closing authority. What remains for this module is the sharpest verification idea in the training: proving controls by watching them say no.