Graph RAG: retrieval that follows relationships.

What is Graph RAG?

Graph RAG is retrieval-augmented generation that uses a knowledge graph as part of its index. During indexing, a language model reads each chunk and extracts entities (people, products, places, concepts) and the relationships between them, which are stored as nodes and edges, often with chunk-level provenance. At query time the system can retrieve by graph neighbourhood, by relationship path or by pre-computed summaries of clusters in the graph, as well as by vector similarity. The result is retrieval that knows that two facts in different documents are about the same thing, which chunk similarity alone cannot see.

How Graph RAG works

Indexing has four steps: chunk the documents; run an extraction prompt over each chunk to list entities and relations with short descriptions; merge duplicates so one entity has one node; and, in the Microsoft variant, detect communities in the graph and write a summary for each at several levels of granularity. Querying has two modes. Local search starts from entities matched to the question and gathers their neighbours, related chunks and summaries. Global search answers corpus-wide questions ('what are the main themes') by mapping over community summaries and reducing them into one answer. Both end with a language model writing from the gathered context.

Difference between RAG and Graph RAG

Plain RAG retrieves chunks that look like the question. Graph RAG retrieves through structure: it can follow 'supplier of' from one company to another to a contract clause, three documents apart, and it can answer questions about the whole collection by reading summaries instead of sampling chunks. Plain RAG is cheaper to index (embed once), simpler to debug and already excellent for 'find the passage that answers this'. Graph RAG costs many model calls per chunk at index time, adds extraction errors as a failure mode, and shines only on multi-hop and global questions. Measure your question set before paying for it.

Difference between vector database and knowledge graph

A vector database stores embeddings and answers 'what is similar to this'. A knowledge graph stores typed nodes and edges and answers 'what is connected to this, and how'. The first is fuzzy and fast and needs no schema; the second is precise and explainable and needs entities to be extracted and reconciled. They complement each other: in Graph RAG the vector index finds entry points (which entities or chunks match the question) and the graph expands from there. Postgres with pgvector plus a relations table, or Neo4j with vector search, can hold both in one place.

Does GraphRAG use embeddings?

Yes. Embeddings are used to match the question to entities and chunks as entry points, to find near-duplicate entities during merging, and in some implementations to embed community summaries for retrieval. What is different is that embeddings are no longer the only signal; the graph structure decides what else gets pulled in. If you remove embeddings entirely you have a classic knowledge-graph question-answering system, which struggles with vague or paraphrased questions. Graph RAG keeps the fuzziness of vectors for finding a starting point and adds structure for everything after.

What is Microsoft Graph RAG?

Microsoft Research published GraphRAG in 2024 as a paper and an open-source Python package. Its distinctive idea is community summarisation: after extracting the entity graph, it clusters the graph hierarchically and asks a model to write a report for each community, so that global questions about a corpus can be answered by reading reports rather than chunks. The package handles extraction, graph building, summarisation and both local and global query modes. It is useful as a reference implementation and for experiments; be aware that indexing a large corpus with it costs many model calls.

When to use Graph RAG

• Questions that need several hops: 'which projects used a supplier that later failed an audit'.
• Questions about the whole collection: themes, trends, 'summarise everything about X across all reports'.
• Corpora with many recurring entities whose relationships matter: contracts, case files, research literature, codebases.
• When plain hybrid RAG has been measured on your question set and fails specifically on relational or global questions.
• Not for: a single manual, FAQ-style questions, or any case where you have not yet measured ordinary RAG. Most student and early product needs end there.

How to build Graph RAG

Start from a working hybrid RAG with a test set, so you can measure what the graph adds. Add an extraction stage: for each chunk, prompt a model to return entities and relations as structured JSON with the chunk ID as provenance. Normalise entity names and merge duplicates (embedding similarity plus a model check works). Store nodes and edges in a graph database or two Postgres tables. At query time, match the question to entities, pull their neighbourhood and the chunks behind it, and add those to the context alongside the usual retrieved chunks. Only then consider community summaries for global questions. Measure after each step; stop when the numbers stop moving.

What is agentic Graph RAG?

Agentic Graph RAG lets an agent decide how to use the graph instead of running a fixed retrieval. The agent reads the question, chooses between local search, global search, a graph traversal or plain vector retrieval, runs it, inspects the result, and may run another step with a refined query before answering. It can also write a graph query directly when the question is precise. This improves hard questions at the cost of latency, more model calls and harder evaluation. Add it only when a fixed pipeline measurably fails, and keep the tool set small so the agent's choices stay predictable.

Is Graph RAG open source?

The main implementations are. Microsoft's GraphRAG package is open source on GitHub. LlamaIndex and LangChain both ship graph-based retrievers and knowledge-graph index types. Neo4j publishes GraphRAG tooling for its database, and lighter alternatives such as LightRAG and nano-graphrag exist for experiments. The graph database itself can be open source (Neo4j Community, or plain Postgres). What is not free is the indexing compute: extracting entities from every chunk with a language model is the dominant cost, so try it on a small corpus first and use a cheaper model for extraction.