Retrieval Respects Permissions

ℹLeadership brief

What it is: the access tiers from corpus design, enforced at the exact point where passages are fetched — before the model ever sees them — with citations re-checking access when opened.

What it buys: the oversharing failure becomes structurally impossible instead of policy-discouraged. A model cannot leak a passage it never received, which is a stronger guarantee than any instruction to a model can be.

What to fund: nothing new — this is a filter clause in the retrieval built last lesson. The cost is design attention, and the alternative was priced in 2.3’s brief: re-indexing under incident pressure.

Prompt first: put the tier in the query path

Extend the retrieval from 3.2. The manifest gives every document an
access tier; every query now arrives with an asker context (role,
tiers held).

Requirements:
  1. The candidate search itself excludes passages whose document
     tier exceeds the asker's - the filter is IN the retrieval
     query, not applied to results afterward, and never applied by
     the model.
  2. A query that loses passages to the filter behaves exactly as if
     those passages did not exist - same ranking behavior, no
     "results were hidden" hint.
  3. Citation opening takes the asker context and re-checks the tier
     against the CURRENT manifest before serving the document.
  4. Show me the test: the same question asked as all-staff and as
     contracts-manager, with the passage lists side by side.

Then list every path a passage could take to the model that does
NOT go through this filter. If the list is not empty, that is the
finding.

Why the filter must live in retrieval, not after it

There are three places people put this control, and two of them fail:

Filtering the answer — let the model see everything, instruct it not to reveal restricted content. Fails structurally: the model’s answer is derived from what it read, and derivation leaks. A summary shaped by the board minute reveals the minute without quoting it, and no instruction reliably prevents what the site’s own security training states as the ground rule — content the model reads is content the model may act on. Instructions are not a boundary.

Filtering the results — retrieve everything, drop restricted passages before the model sees them. Nearly right, and subtly wrong twice: the restricted passages still shaped the ranking (their presence displaces what the asker should have seen), and every code path that touches results-before-filtering is one refactor away from skipping the filter.

Filtering the candidates — the search itself runs over only what the asker may see. The restricted passage is not late-dropped; it was never a candidate. A model cannot leak what it never received — a structural guarantee for the passage-content channel, indifferent to model behavior, prompt injection, or instruction-following. It is the strongest single control in this training, and it governs one channel: what 5.1 adds (aggregation) and the silence rule below (presence) are the channels it does not cover.

The silence requirement

Requirement 2 in the prompt deserves its own paragraph, because it is a real design tension. When the filter removes passages, should the asker be told “some results were withheld”?

No — and 2.3’s map explains why. The existence of a restricted document bearing on this question is itself information at the restricted tier: “results withheld” on a question about a pending acquisition tells the analyst what the board is discussing. The honest behaviors are the ones from 1.4’s eval set: answer from what is visible, or refuse as “not in the corpus” — from this asker’s view, it is not.

The citation loophole

The filter guards question-time. Citations create a second door: an answer served yesterday carries a citation opened today, possibly forwarded to someone else, possibly after the document’s tier changed. So requirement 3: opening a citation is an access check against the current manifest, in the opener’s own context — not a static link to a copy.

This is the same rule the previous training applied to served answers leaving a boundary, now applied to sources: the check travels with the artifact, because the artifact outlives the moment it was authorized.

⚠The audit that makes this real

The prompt’s final demand — list every path to the model that bypasses the filter — is the difference between having a control and believing you have one. Real bypasses found this way: a debug endpoint that queries the raw index; the evaluation harness itself (it queries as nobody, seeing everything); the re-index job logging sample passages into a world-readable log. Each is a path where “in the corpus” quietly becomes “visible.” The list must be empty, or the finding routed.

Stop and escalate when the asker context itself is unavailable — the serving surface cannot say who is asking, or roles do not map to tiers. That is an identity-infrastructure gap, owned by the platform team, and the honest interim is serving only the all-staff classes: shipping restricted classes on an honor-system asker context is 2.3’s incident with extra steps.

KNOWLEDGE CHECK

A restricted passage is retrieved for an unauthorized asker but dropped from the results before the model sees it. The ranking, however, was computed over all candidates. What information can still leak?

Key takeaway

The tier from corpus design becomes real at exactly one point: the candidate search, where a restricted passage is not dropped late but never a candidate at all — a model cannot leak what it never received, and that structural guarantee is indifferent to every model failure mode. Filtered queries stay silent about what they filtered, because existence is information; citations re-check access against the current manifest on every open, because artifacts outlive their authorization. Then audit every path to the model, and route what the audit finds. Lesson 3.4 asks the question all of this was built to answer: did retrieval find the evidence at all?

LEADERSHIP DECISION   accept only the structural control - the filter
                      in the candidate search - and treat any
                      instruction-based alternative as no control
PRACTITIONER ACTION   filter candidates, not results; keep filtered
                      queries silent; re-check access on every
                      citation open; audit every path to the model
SUCCESS MEASURE       the bypass-path list is empty and re-audited on
                      change; the same question asked at two tiers
                      provably sees two corpora