Definition
Knowledge Graph RAG is a retrieval-augmented generation pattern that uses graph-structured knowledge as part of retrieval.
The plain-English version:
retrieve through entities and relationships, not only through similar chunks
Instead of treating documents as isolated text chunks, the system can extract or use entities, relationships, communities, and graph paths to gather connected evidence.
Why This Concept Exists
Plain vector search often works well for direct questions:
What is the refund window?
But it can struggle when the question requires connecting pieces:
Which suppliers are indirectly affected by this policy change?
or:
What themes explain why project delays increased across departments?
The answer may require relationships across documents, not one nearest chunk.
Knowledge Graph RAG exists because some questions are about connected structure.
The Beginner Mental Model
A beginner may think:
Graph RAG means vector search plus a graph database.
That is too implementation-specific.
A better mental model:
documents -> entities and relationships -> graph-shaped retrieval -> grounded synthesis
The graph may help the system find related entities, traverse connections, summarize communities, or assemble evidence that plain semantic similarity would miss.
What Gets Indexed
A graph-based RAG system may index:
- entities such as people, products, projects, policies, or services
- relationships between entities
- claims or facts from source documents
- source text units
- community clusters
- summaries at different levels
- embeddings for text and graph artifacts
The graph does not replace documents. It gives the system another way to organize and retrieve from them.
A Concrete Example
Imagine an internal knowledge base with project updates.
A user asks:
Why did the Atlas migration slip across multiple teams?
Plain vector search may retrieve one update mentioning "Atlas migration."
Knowledge Graph RAG may connect:
- Atlas migration
- database team
- authentication team
- vendor delay
- compliance review
- dependency on shared schema changes
The answer can synthesize across connected evidence instead of quoting one nearest chunk.
Local And Global Retrieval
Graph RAG systems often distinguish local and global questions.
Local questions focus on specific entities:
What is Project Atlas blocked by?
Global questions ask about broader themes:
What are the top operational risks across all migration projects?
Local retrieval may traverse entity neighborhoods. Global retrieval may use community summaries or map-reduce-style synthesis across clusters.
The product should choose the retrieval shape that matches the question.
Failure Modes
Knowledge Graph RAG can fail when:
- entity extraction is wrong
- relationships are missing or hallucinated
- graph construction is too expensive
- summaries hide important source detail
- graph traversal retrieves connected but irrelevant evidence
- provenance from graph facts back to source text is weak
- updates leave graph artifacts stale
Graph structure adds power, but also adds a serious indexing and validation burden.
Common Confusions
Knowledge Graph RAG is not always better than vector RAG.
It helps when relationships and global structure matter.
A graph edge is not automatically true.
Edges must be extracted, stored, and traced back to source evidence.
Graph RAG is not only a database choice.
It is a retrieval and context-assembly strategy.
Community summaries are not source documents.
They may help synthesis, but source grounding still matters.
What This Does Not Mean
Knowledge Graph RAG does not remove the need for vector search, reranking, or context engineering.
Many strong systems combine graph retrieval with lexical, vector, and summarization paths.
It also does not justify claiming private implementation details about real products. Use graph RAG as a recognizable architecture pattern, not a rumor engine.