GraphRAG and Knowledge Graphs, When the Extra Structure Pays

Pull-quote: “A knowledge graph is a second database with a second schema and a second freshness problem. Sometimes the questions justify it. Ask about the questions before you ask about the graph.”

Why this matters

GraphRAG is the most oversold pattern in retrieval right now, and also genuinely valuable. Both things are true, and the difference is the question mix. The pattern: extract entities and relationships from the corpus into a graph, summarize communities of related entities, and answer questions by traversing structure instead of (or alongside) matching text. For some corpora and question shapes this unlocks answers nothing else can produce. For most, it is an expensive way to rebuild hybrid retrieval with more moving parts.

What the graph actually buys

Two capabilities, specifically:

1. Multi-hop traversal. “Which suppliers connect to programs affected by directive X?” is a path query. Text retrieval finds passages about each entity; the graph is the join. When the answer is a path rather than a passage, the graph answers a question the vector index cannot represent.
2. Corpus-global summarization. “What are the recurring themes across these ten thousand reports?” has no relevant top-k, because every document is slightly relevant. Community detection plus per-community summaries gives the model a pre-computed map of the whole corpus, which is the only honest way to answer global questions without pretending twenty retrieved chunks represent ten thousand documents.

If your question logs contain neither shape in meaningful volume, stop here; hybrid retrieval with good metadata filtering will serve you better for a fraction of the cost.

The honest cost model

The build cost is one large LLM bill. The maintenance cost is a standing tax that grows with corpus churn, and a stale graph is worse than no graph, because traversal results carry an authority that retrieved text does not.

The hybrid pattern

Teams that make GraphRAG work rarely replace vector retrieval; they route between structures:

   Question ──► Router
                 │
        ┌────────┼────────────────┐
        ▼        ▼                ▼
   Vector +   Graph traversal   Community
   BM25       (entity, path,    summaries
   (passage    join questions)  (global "themes
    questions)      │            across corpus")
        │           ▼                │
        │      passages linked       │
        │      to path nodes         │
        └───────────┬────────────────┘
                    ▼
             Answer + citations

The graph finds which entities and paths matter; the text index supplies the evidence passages attached to those nodes, so answers stay cited against source documents rather than against extracted edges. Traversal for structure, text for grounding, each component doing the one thing it is good at.

A decision rule

Sample your real query logs. If path-shaped and corpus-global questions are rare, the graph is a science project. If they are common (regulatory networks, supply chains, safety event chains, investigative corpora), scope the graph to the entities those questions actually traverse, and keep passage-level citation mandatory. The bar is the same one that applies to any retrieval structure: it earns its maintenance cost only when the question mix demands it, and the cited passage remains the unit of trust either way.

Closing

GraphRAG is not the next generation of RAG; it is a specialized index for two question shapes, paths and global summaries, that plain retrieval handles badly. Price the extraction, the entity resolution, and the standing freshness tax against the real frequency of those questions in your logs. When the math works, build the graph scoped and hybrid. When it does not, boring hybrid retrieval with good metadata is not a compromise. It is the right answer.