Multi-tenant RAG: making sure one customer never sees another customer's documents

A shared vector index serving multiple customers needs strict isolation, not just a metadata filter that a bug could silently bypass. What actually guarantees separation.

Written by

Pranjul Rathour

GenAI Engineer at SCULT INDIA, based in Kanpur, India. Builds production RAG systems, fine-tunes open LLMs and ships agentic AI products; has mentored 200+ college developers. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks.

pranjulrathour41@gmail.com LinkedIn GitHub X Instagram Bluesky Dev.to

In a packed college auditorium

A SaaS product built on RAG, serving multiple customers from one system, has one non-negotiable requirement: customer A must never retrieve customer B's documents. A metadata filter alone is a fragile way to guarantee this — one bug in the filter logic and the guarantee is gone.

Stronger isolation options

Separate indexes or namespaces per tenant, where the vector store itself enforces the boundary structurally, not just via a query filter.
If sharing an index, enforce the tenant filter at the database or API layer, never trusting application code alone to always include it correctly.
Test isolation explicitly: a test that logs in as tenant A and tries to retrieve tenant B's known documents should be part of your test suite, not an afterthought.

Why this deserves more caution than most RAG advice implies

A retrieval bug that leaks one wrong chunk into an answer is embarrassing. A retrieval bug that leaks one customer's confidential documents to another is a genuine incident. Design the isolation boundary as if a filter bug is inevitable, not unlikely.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.