Why coarse-grained permissions break in RAG pipelines

Fine-grained access control can be difficult. Say, for example, that you've built a retrieval augmented generation (RAG) pipeline on Elasticsearch for an internal knowledge base. The requirements were reasonably straightforward: an AI chatbot front end that retrieves answers from 100K documents chunked into dense_vector embeddings. Permissions were fairly trivial, too, consisting of just a single department keyword filter in the kNN query.

Real-world requirements are often far more complex. Maybe the engineering team wants to share just one architecture document with the sales team, and one exec needs a personal exception to a document outside their department. How would you solve this in your RAG pipeline?

A stopgap solution can quickly become unwieldy. You could create a new department for each request, but then you end up with an explosion of roles that go stale the minute your system changes. You could also add a bunch of if-else statements, but that's a bad idea. Encoding security decisions in application code, which is neither auditable nor scalable, is how data breaches occur and is something you should protect against. This is especially true in enterprises and compliance-regulated industries, such as finance and public sector. 

The latest OWASP list for LLM risks points out several industry-wide vulnerabilities specific to LLM applications, including: 

• Sensitive Information Disclosure.

• Excessive Agency.

• Vector and Embedding Weaknesses.

So how do you add fine-grained permissions to your RAG pipeline? The answer lies in relationship-based access control (ReBAC).

ReBAC to the future: How relationship-based access control works

In 2019, Google published Zanzibar: Google's Consistent, Global Authorization System, which describes the authorization system behind hundreds of Google services, including Calendar, Drive, Maps, Photos, and YouTube. If you've used any of these services, your requests have most likely been checked by Zanzibar.

The paper made waves in the world of identity and access management (IAM). It’s also widely credited with popularizing ReBAC, which was a departure from the models that most teams used at the time, including role-based access control (RBAC) and scoped tokens.

The core idea behind ReBAC is that permissions are determined by the relationships between subjects (users, agents, groups) and resources (documents, folders, projects). You can store those relationships as a graph: nodes are the objects, and edges are relationships like "viewer," "editor," or "owner." A permission check then boils down to a single question:

Instead of force-fitting an access decision into a role, you answer that question by checking whether a path exists in the graph.

Here's a real-world example. When user:jordan creates document:somedocument in Google Docs, Zanzibar records that Jordan is a writer of the document as a relationship tuple:

document:somedocument#writer@user:jordan

If Jordan then shares the document with user:alex as a reader, another tuple is created:

document:somedocument#reader@user:alex

There's one more rule: every writer is also a reader. Conceptually, Zanzibar now sees a graph like this:

When Jordan tries to read somedocument, the system looks for a directed path from the document's reader relation to Jordan. There's no direct reader edge for Jordan, but reader includes writer, and Jordan is a writer. Now that the path is found, Jordan can read the document. If there were no path, Jordan would see the "You need access" screen that we all dread.

Now it's your turn. Can Alex write to the same document?

The next time you open (or can't open) a file in a Google product, you'll know what's happening behind the scenes.

What does this have to do with my RAG pipeline?

Zanzibar was designed for high-throughput, low-latency authorization checks over hierarchical groupings of organizational data. As it turns out, AI applications, such as agents and RAG pipelines, need exactly that. Look back at the requirements at the top of this article. Coarse-grained approaches, such as RBAC or scoped tokens, can't bend to per-document sharing or cross-department access, nor can they accommodate one-off exceptions at scale. The relationships between users and the resources they can access form an ever-evolving graph, which is exactly the problem ReBAC solves.

But unless you work at Google, you can't actually use Zanzibar. Fortunately, there are open source implementations of it. We're going to use SpiceDB, named after Dune’s famous line, “The spice must flow.” The techniques mentioned here aren’t RAG-specific; they can also be used to add permissions to your enterprise applications and AI agents, in addition to any other resources that you want to protect.

Define a SpiceDB schema, relations, and permissions

As the name suggests, ReBAC is built on the relationships between the objects in your system. So, when adding permissions to your application, the first step is to define a SpiceDB schema.  This describes the types of objects in your application and how those objects can relate to one another. It also defines the permissions derived from those relationships. For our RAG pipeline, it looks like this:

There’s a lot to unpack here:

• The three object types are user, department, and document.

• The relations are as follows:

• A user can be a member of a department.

• A user can be the owner of a document.

• A user can be a viewer of a document.

• You can view a document if you're a viewer or its owner. And viewer permissions can be granted in one of two ways: to a specific user or to every member of a department (department#member).

• Only an owner of a document can edit it.

The schema represents the shape of your system, and it rarely changes. 

In SpiceDB, a working permissions system is the combination of a schema and relationships. A relationship is a specific instance of a relation. For example, "user jordan is an owner of document readme" is written as a relationship tuple:

document:readme#owner@user:jordan

What if you wanted to say that alice is part of engineering, and everyone in engineering can view eng-arch-001? That's also just relationships:

• department:engineering#member@user:alice ("Alice is a member of engineering.")

• document:eng-arch-001#viewer@department:engineering#member ("Every engineering member can view this doc.")

So if alice clicks a link to view eng-arch-001, SpiceDB walks the graph like this:

If no path exists, access is denied.

That's how a permission check works in SpiceDB. Before performing any action, you call the CheckPermission API:

We'll use this same check in our RAG pipeline to make sure that users only get answers based on data they're allowed to see.

When access rules change, write a relationship not a role

Remember how the requirements for our RAG project changed? In a role-based system, that would mean refactoring your roles or burying more authorization logic in your code. With ReBAC and SpiceDB, every new access rule is just a new relationship:

RBAC / scoped tokens

ReBAC with SpiceDB

Access expressed as

Roles assigned to users

Relationships between subjects and resources

A permission check asks

Does this user hold the role?

Does a path exist in the graph?

One-off exception for one person

New role

One relationship write

Sharing with another team

New role per combination

One relationship write via department#member

When access changes

Refactor roles or edit filters

Write or delete a relationship

Where the logic lives

Application code

SpiceDB

If the engineering team wants to share a document with the sales team, you add this relationship:

document:eng-arch-001#viewer@department:sales#member

And you can create an individual exception for an exec, like so:

document:sales-proposal-001#viewer@user:ceo

In RBAC, you'd be inventing new roles or rewriting filters, but in SpiceDB, each of these is a single WriteRelationships call. Here's the sales share:

This requires no new roles or code changes, and it needs no reindexing. OPERATION_TOUCH makes the write idempotent; that is, it creates the relationship, or, if it already exists, leaves it alone.

Wire SpiceDB into your Elasticsearch RAG pipeline

Let's bring this all together. When you index chunks into Elasticsearch, you store metadata alongside each embedding that says which document it came from. In our repo, that's a doc_id field. It's the same ID that SpiceDB uses for the document resource, and it's what connects the two systems.

Here's the query flow:

1. The user's query goes through an embedding model (Mistral, in our repo) to generate a query embedding.

2. Elasticsearch runs a kNN search to retrieve the most relevant chunks. 

3. Before our documents are passed to the LLM, we send each chunk's doc_id and the user's ID to SpiceDB.

4. We drop every chunk that SpiceDB determines the user can't view, based on the defined rules.

5. The permitted documents are passed into the prompt. In the case of agentic RAG, the relevant documents are passed as the output of the tool to the LLM.

This pattern is called post-filter authorization: retrieve first, and then filter. The check happens before the LLM to ensure that unauthorized documents aren’t passed via the context window. That's the security boundary. Anything that reaches the model can end up in its answer.

Here's the retrieval step (abridged) from the repo:

Notice that there's no permission logic in the query. Elasticsearch finds the relevant content, and SpiceDB decides who gets to see it.

Next comes the authorization step. Here's the repo's authorization_node. It hands the retrieved documents to SpiceDBAuthorizer from the langchain-spicedb package, which checks all of them against SpiceDB in a single CheckBulkPermissions call:

_get_authorizer() sets the authorizer up once to check the view permission on document resources for user subjects, using each document's doc_id metadata as the resource ID. Only documents with a confirmed path in SpiceDB come back in result.authorized_documents. A document with no doc_id, or an individual check that errors, is dropped. If SpiceDB can't be reached at all, the call raises and the pipeline stops before generation.

What about Elasticsearch document-level security?

Elasticsearch has document-level security built in, and it's a great fit when access can be expressed as a query over fields in the document, like department: engineering. It gets more difficult when access depends on relationships like the ones mentioned in the introduction, such as per-user exceptions or sharing through groups. Other examples include access inherited from folders and orgs. You'd have to denormalize that graph into every document and map those to defined RBAC roles. You’d also have to reindex whenever a share changes. SpiceDB keeps the graph in one place and gives the same answer to your RAG pipeline and your app, as well as to your agents.

Never let the agent decide whether to authorize

The pipeline follows a clear shape: retrieve → authorize → generate (with an optional reasoning loop), built as LangGraph nodes.

The core rule is simple: the agent may reason about what to retrieve or what to generate. It may even reason about how to explain a denial. But it must never reason about whether authorization applies.

LLMs are inherently probabilistic. If the model decides whether to check permissions, a well-crafted prompt injection can persuade it not to do so, thus leaking data. Enforcement lives outside the model in a deterministic node that the agent can't route around. That node must also block access by default in the event of a failure. An error or a missing path means no access.

Here's how the three nodes are wired up, with an entry point, the edges, and an endpoint using LangGraph:

There's no edge from retrieve to generate. The only road to the LLM runs through authorize. The repo adds an optional reason node for retries (the agentic part), but every path still passes through authorization first.

Each LangGraph node is just a function. You saw authorization_node in the previous section, and here's generation_node:

The generation node reads only authorized_documents, so denied content never reaches the prompt. GENERATION_PROMPT tells the model to answer only from those documents. It also passes along the denied count, so the answer can point out that some information wasn't accessible (instead of pretending it doesn't exist). It’s important not to return details about rejected documents, since those details could leak  information that the user isn’t authorized to access.

Taking fine-grained authorization to production

How do you go from a proof of concept to a production-grade implementation? Here are our recommendations:

• Overfetch to recover post-filter recall. Post-filtering can leave the LLM short on context. For example, the user can ask about “microservices patterns,” and if four of the top five hits point to documents they can’t see, the LLM has a single document to work with. The fix is to overfetch: retrieve k × N candidates (where k is how many documents you want and N is a multiplier that you tune), and run the permission check. Then, keep the first k that survive. Note that overfetching improves the odds but doesn’t guarantee k survivors. num_candidates only has to change if the new k would exceed it; if you leave it unset, it defaults to 1.5 × k (or 1.5 × size when k isn’t set) and scales on its own. num_candidates is capped at 10,000, so overfetching has a ceiling. If a user can see so little of the corpus that you’d need more than that, SpiceDB has a LookupResources API which finds all resources of a given type that a specific subject can access. It works best when the result set is moderate: SpiceDB’s docs warn that it gets expensive above roughly 10,000 results. In that case, the docs recommend post-filtering with CheckBulkPermissions. You’d typically use this API to get the list of document IDs that the user can access and then pass them as a terms pre-filter in the kNN query’s filter parameter (up to the default index.max_terms_count value of 65,536 terms).

• Add a coarse-grained metadata pre-filter. Authorization is rarely the only constraint in a RAG pipeline. You may also want to restrict results by department or recency or by classification. Elasticsearch's kNN filter applies those conditions during the vector search itself, so you don't waste top-k slots on documents that you'd throw away. SpiceDB still makes the fine-grained call afterward:

• Keep relationships in sync with the source of truth. In the repo, a setup script seeds the relationships; but in production, they come from wherever access is actually managed, like your document system or HR directory. They could also come from your identity provider. Write a relationship when a document is shared or someone joins a team, and delete it when access is revoked. Because permissions live in SpiceDB and not in the index, none of this requires reindexing.

• Deploy SpiceDB and Elasticsearch. The repo runs SpiceDB and Elasticsearch in local Docker containers, with SpiceDB storing relationships in memory. A production deployment of SpiceDB needs a persistent datastore, like PostgreSQL. You also need to run SpiceDB somewhere. There are currently three main deployment options:

• SpiceDB, self-hosted (open source). Run and manage it entirely on your own infrastructure.

• AuthZed Cloud. Managed, self-service, pay-as-you-go SpiceDB with built-in enterprise features, like audit logging.

• AuthZed Dedicated. A private, single-tenant deployment provisioned in your chosen cloud provider and region.

Similarly, Elasticsearch can be deployed alongside, via Elastic Cloud Hosted, Elastic Cloud Serverless, and on-prem distributions, with the options outlined in the documentation.

Elasticsearch retrieves, and SpiceDB authorizes

Elasticsearch finds the most relevant content, and SpiceDB decides who's allowed to see it. A deterministic authorization node makes sure that the user gets responses based only on the information they’re allowed to access.

To try it yourself:

1. Clone the sample repository, and follow the README to start SpiceDB and Elasticsearch with Docker Compose.

2. Ask the same question as two different users, and compare what each one gets back.

3. Read the annotated Zanzibar paper.