The Council Pattern
Updated
AdvancedWhat you'll learn
~18 min- Recognize when a change deserves independent multi-model review
- Run a diff through a council and read structured verdicts
- Weigh council cost against the cost of being confidently wrong
You have a schema migration staged. The agent wrote it, reviewed its own work, and says it is safe. Before you run it against production, you pipe the diff to three independent models. One says yes, one says tradeoff, and one says no — this drops a column with data in it.
That is the council pattern in one sentence: ask models that were not involved in writing the change to vote on whether it should ship.
- Typed in the terminal:
llm-council estimate --mode review --diff "Is this migration safe to run?" - The tool replies: mode: review participants: claude, codex, antigravity estimated_prompt_tokens: 8400 budgeted_rounds: 1 claude, codex, antigravity: cli (billed to your existing subscriptions, n/a here) known_total_usd: $0
- Typed in the terminal:
llm-council run --mode review --diff "Is this migration safe to run?" - The tool replies: Reading diff (staged and unstaged changes)...
Peer 1 (claude)
RECOMMENDATION: yes
Reasoning: Column rename is backward-compatible. Application code updated in same commit. Index preserved.
Peer 2 (codex)
RECOMMENDATION: tradeoff
Reasoning: Safe if deployed atomically with app code. Rolling deploy without feature flag risks 503s during transition window.
Peer 3 (antigravity)
RECOMMENDATION: no
Reasoning: Line +14 drops
user_legacy_id. Column is referenced in audit_log trigger (not in this diff). Data loss on next migration run. ───────────────────────────────────── Verdict summary: 1 yes / 1 tradeoff / 1 no Council did not reach consensus. Investigate the dissent before committing. Runllm-council lastto view full transcript.
Illustrative session; exact output varies by tool and agency configuration.
One peer caught a trigger reference that was not in the diff. The agent that wrote the migration did not catch it — not because it is a bad model, but because it shared the blind spot with itself. The peer closest to the agent that wrote the change, same vendor and model family, is the least independent check on it; a council is stronger with peers from more than one vendor for that reason.
Why independent review works
A single model reviewing its own output has a structural problem: it inherits its own reasoning path. If it decided early that a column was unused, it will tend to re-confirm that belief when asked to double-check. This is not hallucination — it is the same confirmation bias that makes self-review weaker than peer review in any engineering process.
Independent models bring different training distributions, different prompt paths, and different tendencies toward caution. A mistake one model produces with confidence is exactly the kind of thing a second model, trained differently and reasoning from a fresh start, is positioned to catch. A dissenting review is more informative than an agreeing one: agreement can mean the change is genuinely fine, or it can mean every reviewer shares the same blind spot; dissent tells you where to look. In practice three reviewers in one round covers most changes; add peers or a second round only when the change is risky enough to justify the extra cost, which grows with every peer and round. Two peers is often enough for a quick sanity check; one is not a council.
This matches how consequential decisions work in every high-stakes domain: peer review, second opinions, red teams. The pattern is not new. What is new is that it is now automatable in a pre-commit hook.
When to convene — and when not to
Convene a council when the cost of being confidently wrong is high:
- Schema migrations — dropped columns, renamed foreign keys, changed constraints
- Auth and security diffs — anything that touches tokens, session handling, permissions, or secrets
- Architecture decisions — new service dependencies, breaking API changes, deployment topology shifts
- Irreversible operations — data backfills, purges, vendor lock-in adoption
- “The agent is confidently insisting” moments — when the model has argued itself into a corner and is pushing hard for a direction you cannot fully evaluate
Before any of these, run the review your tool already ships. It is the first reviewer on every diff, and the council is for the risky ones above (commands as of 2026-11-07):
- Claude Code:
/code-review(/reviewis an alias)./code-review ultraruns a cloud multi-agent review, with three free runs on Pro, Max and Ultra and usage credits after that. Claude Code’s help calls it “a cloud-hosted multi-agent code review”: several reviewers, but all from one vendor, so it is not the second vendor’s opinion a council adds. - Codex CLI:
/review, then choose the base branch, uncommitted changes, a commit, or custom instructions. - Copilot CLI:
/review, and/security-reviewfor a security pass. - Antigravity CLI: no review agent is documented; open
/diffand ask for a review in plain words.
Do not convene a council for:
- Routine edits — a council on every commit is ceremony, not engineering
- Style and formatting changes
- Documentation updates
- Green-path feature work where a simple test covers correctness
The investment framing matters. A review mode council costs a little in tokens and runs while you do something else — minutes for a small diff, longer when peers are reading a large one. A missed schema bug that reaches production can cost days of recovery, data repair, and incident retrospectives. The council is not overhead — it is insurance with a known premium and a calculable expected value.
Perplexity shipped Model Council in February 2026, first for its Max subscribers on the web. Council-style code review circulates as a first-class pattern in the Claude Code ecosystem. This course’s own production workflow uses multi-model deliberation before non-trivial changes merge to main. The pattern has left early-adopter territory.
Hands-on: running a diff review with llm-council
llm-council (MIT license, built and maintained by this course’s author, Intellimetrics) is one concrete implementation of the council pattern. It convenes your locally installed CLI tools (claude, codex, antigravity as the agy binary, and copilot) and, optionally, hosted API models or local Ollama models as independent peers. The tool is the demo; the pattern is the lesson. Everything in this section is grounded in the README at github.com/Intellimetrics/llm-council.
The hands-on below convenes two or more installed CLIs, signed in on plans you already pay for; the openrouter and private-local presets further down need no second CLI.
Install
# Recommendeduv tool install --force git+https://github.com/Intellimetrics/llm-council.git
# Alternativepipx install --force git+https://github.com/Intellimetrics/llm-council.gitAfter install, run setup and diagnostics:
llm-council setup --yes --preset auto # detect installed CLIs, configure peersllm-council doctor # verify all peers are reachableThe auto preset discovers which CLIs you have installed. Other presets (tri-cli, openrouter, tri-cli-openrouter, private-local) are covered below.
The estimate-before-running habit
Before any council run, get a cost estimate:
llm-council estimate --mode review --diff "Is this migration safe to run?"This shows the prompt size and, for peers billed per token (hosted API models), estimated USD, without sending anything. Native CLI peers run on your existing subscriptions, so the estimate lists them without a price. Build this into your workflow: estimate, decide whether the change is worth it, then run. If you have added API peers, a large diff in consensus mode is where the dollar figure climbs: the signal to trim the diff or decide the change warrants the spend.
Review mode: the standard pre-commit gate
Make your changes (staged or not), then:
llm-council run --mode review --diff "Is this migration safe to run?"--diff adds your uncommitted changes (staged and unstaged) to the prompt automatically. Each peer independently reads the diff and returns one of three structured verdicts:
RECOMMENDATION: yes # safe to proceedRECOMMENDATION: no # stop — major issues detectedRECOMMENDATION: tradeoff # plausible, but critical trade-offs notedEvery peer is asked to end on one of these labels, and the tool checks for it, so a reviewer has to commit to a position, which is exactly what you want from a reviewer.
After the run, the full transcript is available:
llm-council lastReading a split verdict
A split result — the scenario in the terminal above — is not a failure. It is the council doing its job. When you get 1 yes / 1 tradeoff / 1 no, the correct response is not to average the votes. It is to read the no peer’s reasoning first. A single dissenting peer that names a specific artifact you did not consider has more informational value than two agreeing peers who did not catch it.
Weight dissent over agreement. Investigate every no and every tradeoff before proceeding. If the no peer is wrong, you will know after thirty seconds of reading — and you will have documented why you shipped anyway.
Trust properties
Read-only enforcement — the honest version
When native CLI peers run, their write tools are disabled at the flag level: claude launches with its tool set restricted to --tools Read,Grep,Glob,LS (plus --permission-mode manual and no MCP servers), codex with --sandbox read-only, and antigravity with --mode plan inside its OS sandbox (--sandbox). Because peers run headless, any tool call that would normally pause for your approval is denied outright. A peer can read your diff and reason about it but cannot modify files, and even a prompt-injected diff can’t talk it into writing. (This lesson does not cover how the copilot peer is restricted; read that in the tool’s docs before you rely on it.) Hosted and local API peers (OpenRouter, Ollama) are plain API calls with no filesystem access at all.
When this lesson was first written (June 2026), the tool could only enforce read-only on Antigravity at the prompt level. That gap is closed, but the lesson it taught still stands: when you adopt any council tool, read how each peer’s read-only guarantee is enforced — flag, sandbox, or prompt — and don’t assume they’re equal.
Secret scanning
Before any diff leaves your machine, the tool scans it for common secret patterns: API keys, tokens, private key blocks. By default it warns you and counts the matches without printing the values; set secret_scan: block under defaults in .llm-council.yaml to stop the run instead. This is a pre-flight check, not a substitute for .gitignore hygiene and a secrets manager, but it catches the most common accidental-commit scenario.
Cost caps
Add --max-cost-usd to any run to enforce a hard ceiling:
llm-council run --mode consensus --diff --max-cost-usd 0.50 "Should we merge this auth rewrite?"If the estimated cost exceeds the cap, the run is refused, and so is a run that includes a paid API peer whose price the tool cannot look up. Native CLI peers bill to your subscriptions and do not count toward the cap. Useful in CI environments where you want council coverage but not unbounded spend.
If your code cannot leave your machine — classified systems, HIPAA-regulated data, proprietary algorithms — use the private-local preset, which routes all peers through Ollama models running locally. No tokens leave the machine. Quality is lower than frontier models, but the pattern still works: independent local models catch different failure modes than the single agent that wrote the change.
Setup: llm-council setup --yes --preset private-local after installing Ollama with at least two different model families pulled.
Consensus mode: when yes/no is not enough
Review mode gives you a verdict. Consensus mode gives you a debate.
llm-council run --mode consensus --diff "Should we merge this auth rewrite?"In consensus mode, each peer is assigned a stance at the start: one argues for merging, one against, one stays neutral. Each answers once from its stance; add --deliberate to buy a second round when they disagree. The output is not a single verdict but a transcript of positions (and rebuttals, when you add --deliberate), surfacing the strongest arguments on each side. Use it when you need to understand the shape of the risk, not just whether to proceed: architecture decisions, security rewrites, vendor adoptions where the correct answer is genuinely uncertain.
Consensus mode runs longer than a review run (peers get twice the time limit), and more again with --deliberate. Estimate first.
As an MCP server
If you run llm-council as an MCP server, your main agent can call it mid-session without leaving the terminal. The main tools are:
council_run — run a council synchronouslycouncil_estimate — estimate cost before runningcouncil_recommend — should this question go to council at all?council_list_modes — see the configured modes and peerscouncil_doctor — verify peer connectivitycouncil_last_transcript — retrieve the previous runcouncil_models, council_stats, council_query_transcripts and council_config cover catalogs, history and settings.
This enables a workflow where the agent completes a dangerous operation, pauses, calls council_run on the staged diff, and waits for a consensus before proceeding to commit. The human is still in the loop for the final decision, but the council has already surfaced the second opinion.
Your agent has staged a change that renames a foreign key column across five tables. When should you convene a council?
The council returns: Peer 1 — yes, Peer 2 — yes, Peer 3 — no (names a specific trigger you didn't know existed). What is the correct next step?
How does llm-council keep its native CLI peers (claude, codex, antigravity) read-only?
Key takeaways
- The blind-spot argument: a model reviewing its own work inherits its own reasoning path. Independent peers catch different failure modes.
- When to convene: schema migrations, auth/security diffs, architecture decisions, irreversible operations, and confident-agent moments. Routine edits do not need a council.
- Investment framing: a review run on CLI peers draws on subscriptions you already pay for and takes minutes. A production incident costs days. The council is insurance with a calculable premium.
- Estimate before running:
llm-council estimate --mode review --diffbefore everyrun. Make it a reflex. - Verdicts are structured:
yes,no, ortradeoff— no hedging accepted. Weight dissent over agreement; investigate everyno. - Know how read-only is enforced: llm-council disables write tools at the flag level for every native peer (
--tools Read,Grep,Glob,LSfor Claude,--sandbox read-onlyfor Codex,--sandbox --mode planfor Antigravity). When you evaluate any council tool, check the mechanism, not the marketing. - Consensus mode (
--mode consensus) gives each peer an assigned stance for one round (--deliberateadds a second when they disagree) when you need to understand the shape of the risk, not just a go/no-go. - As an MCP server, the council is callable mid-session — your agent can pause, take the current diff to council, and wait for a verdict before committing.
Up next: Lesson 9 — Agent Runbooks — making these patterns repeatable across teams by encoding them as executable operational documents.