Mostly Isolated Is a Breach with a Delay

Before anyone was saying "RAG", I built the boring version of the same problem.

For a legal-tech startup, I had to guarantee something simple to state and easy to get wrong: every document belongs to a tenant, and only the members that tenant has authorized can ever touch it — under an access model we agreed with the business, not one I invented. Documents in, authorization enforced, nobody sees what isn't theirs.

That's the entire problem. Retrieval-augmented generation doesn't change it. It just adds a layer that makes getting it wrong far easier to do, and far harder to notice.

Why semantic search makes a soft boundary dangerous

Classic access control has a merciful property: it's binary and visible. A query returns your rows or it returns nothing. The boundary is a WHERE clause, and when it goes missing, the result usually looks obviously wrong.

Semantic retrieval removes that mercy. The retriever ranks documents by similarity to the question, and similarity does not care whose data it is. Put two tenants' documents in one index, and the moment another tenant's contract is a closer match to the query than your own, it ranks higher. It gets retrieved. It gets handed to the model as context. And the model answers — fluently, confidently, with a citation — using text the user was never allowed to see.

There is no error. No 403. No stack trace. Just a helpful-looking answer built on someone else's data: a leak wearing the costume of good UX.

Isolation is a pre-filter, not a ranking signal

The instinct is to fix this with better ranking: boost the user's own documents, penalize everyone else's. Don't. A boundary enforced by scoring is a boundary you are hoping holds — and hope is not a security control. Tune the weights wrong, onboard a tenant whose data is unusually similar to a neighbor's, swap the embedding model six months from now, and the leak quietly returns.

Isolation is not a property of how results are scored. It's a property of what gets searched at all.

Enforce it before retrieval, at the index layer. Concretely, that's one of three things: a separate index per tenant, a per-tenant namespace inside a shared index (the multi-tenancy primitive in stores like Pinecone), or a metadata filter — tenant_id == acme — applied before the nearest-neighbor search runs, pre-filtering the candidate set rather than scoring results after the fact. The retriever should never lay eyes on another tenant's vectors in the first place. Then similarity is free to do its job, because everything it can see is already yours. You cannot rank your way to isolation. You can only make sure the search never sees what isn't yours.

The filter is your authorization model, not just a tenant ID

One index per tenant is the easy half. The half that bites is inside the tenant.

At that legal-tech company, the rule was never just "firm A can't see firm B." It was "only the members this firm authorized can touch this matter" — a partner sees the whole case, external counsel sees one folder, a paralegal sees what they were assigned. That authorization model was the boundary; the tenant was only its coarsest layer.

RAG flattens all of it by default. Chunk a tenant's documents, embed them, and the retriever will happily surface a chunk from a matter the querying user was never granted — same tenant, wrong permission. A per-tenant index does nothing here: the leak is inside the index.

So the pre-filter can't just be tenant_id == acme. It has to be the user's full authorization predicate, resolved from the authenticated identity at query time and applied before the search: the set of documents this specific person is allowed to see, right now. Isolation isn't a boundary you set once at ingestion. It's your access-control model, evaluated on every retrieval — because permissions change, and the index doesn't.

And it has to fail closed

Where the boundary lives has a twin question: what happens when the code forgets it.

That metadata filter is the dangerous one. If isolation depends on the application remembering to pass tenant_id == acme on every single query, then your data separation is one forgotten parameter away from gone — and that mistake fails in the worst possible direction. An unfiltered search doesn't return an error. It returns everyone's documents. Silent, and maximally harmful: the two properties you least want in a failure mode.

Put the boundary somewhere a mistake fails closed instead. A per-tenant index or namespace scopes the search to one tenant's vectors, so a forgotten scope returns nothing from the others rather than everything. Better still, push it below the query entirely: keep the embeddings in your own database as pgvector rows under Postgres row-level security, and the boundary is enforced on every read regardless of what the query remembered to ask for — and the policy can carry the authorization predicate, not just the tenant. Forget the WHERE and the policy still holds; forget to set the identity at all and it matches nothing — so you get an empty, obviously-broken answer, the kind someone reports in five minutes, instead of your whole customer base served to a single user.

Returns nothing is a recoverable bug. Returns everyone is a breach. Architect so your mistakes land on the first one.

A test that tries to breach it

Enforcing the boundary once isn't the same as knowing it still holds after the next refactor, the next embedding-model swap, the next engineer who "simplifies" the query. So test the breach directly: seed one tenant with a sentinel document, run another tenant's identity through the full retrieval path with a query written to match that sentinel, and assert it never comes back. Wire that into CI, and the day someone drops the filter or trusts the wrong default, the build goes red instead of a customer going quiet.

It's the same move as any fitness function: don't test that the happy path works — test that the thing you're afraid of can't happen.

The test is your build-time proof. Its runtime counterpart is an audit trail of which documents were retrieved for which identity — because a failure this silent leaves no other evidence that it happened. In a regulated domain, that log isn't observability. It's the record.

The same lesson, one layer down

If this feels familiar, it should. A tenant boundary you enforce in application code is a boundary you are trusting every future query to remember — every query a tired engineer writes at 6pm on a Friday, and every query an AI assistant generates without ever having heard of your tenants. Enforce it in the index, or in the database's own access rules, and there is nothing left to remember. The boundary stops being an instruction and becomes a wall.

It's the same principle as enforcing an architectural rule in the build instead of a wiki page: an invariant that protects money, trust, or compliance does not belong in a convention that someone — or something — has to choose to honor.

The rule I'd leave you with

In a system that answers questions over someone's data, "mostly isolated" is not isolated — it's a breach with a delay. Semantic search will surface the neighbor's record eventually; the only open question is whether your architecture made that possible.

Put the tenant boundary where the search cannot run without it, and where forgetting it returns nothing instead of everyone. Everything above the retrieval layer — the model, the prompt, the reranker — is a comfort, not a control.

The retrieval path is the leak I've focused on here; caches, logs, and prompt assembly deserve the same discipline — but that's another piece.

Where in your retrieval path is isolation a filter you apply, rather than a wall the index simply is?