The Knowledge Graph Returns, Questions RAG Cannot Answer
Pull-quote: “A vector index can find every document that mentions two companies. It cannot name the three intermediaries that connect them. That answer is a path, not a passage.”
Why this matters
Vector RAG answers questions whose evidence is a passage: some chunk, somewhere, contains the answer, and the job is to find it. A distinct class of questions has no such chunk. How is supplier X exposed to event Y? What depends, transitively, on this component? Which organizations recur across these incidents under different names? The evidence for these answers is assembled across documents, through relationships, and the answer is a path or a subgraph, not a paragraph. Iterative retrieval can sometimes chain its way there, hopping from result to query to result, but reliability decays with each hop and the cost compounds. When relational questions are the product rather than the exception, the robust move is to make the relationships explicit: extract entities and relations into a knowledge graph and traverse it.
Entity resolution is the real work
The graph conversation usually starts with extraction and schema. The part that decides whether the graph is useful is less glamorous: entity resolution. Real corpora refer to the same entity a dozen ways: abbreviations, transliterations, former names, subsidiaries, a ticker in one source and a legal name in another. Without resolution, the graph is confetti: one real-world actor shattered into five nodes, every path through them broken, every aggregation undercounted.
Resolution is a pipeline, not a step: generate candidate matches cheaply by blocking on names and attributes, decide matches with deterministic rules plus embedding similarity plus shared context, then merge into a canonical node. Two disciplines keep it trustworthy: record provenance for every merge, and keep merges reversible, because a wrong merge quietly poisons every multi-hop answer that crosses it.
Graph for structure, vectors for evidence
The production pattern is not graph instead of vectors; it is a division of labor. Link the question’s entities to canonical nodes, traverse typed edges to find the structural answer, then anchor vector retrieval on the entities and edges along the path to collect the supporting passages a human will actually read.
Question: "How is supplier X exposed to event Y?"
│ entity linking: X, Y ──► canonical nodes
▼
(X) ──supplies──► (component C) ──used_in──► (product P) ──affected_by──► (Y)
│ path found by traversal
▼
Vector retrieval anchored on X, C, P, Y ──► supporting passages
▼
Answer = the path + the cited evidence for each edge| Question shape | Vector RAG alone | Graph + vectors |
|---|---|---|
| “What does the document say about X?” | Strong | Graph adds little |
| “How is A connected to B?” | Co-mention luck | Traversal returns the actual path |
| “What depends on X?” | Misses transitive dependencies | Typed edges enumerate the closure |
| “Which entities recur across these events?” | Duplicates split by name | Resolved nodes aggregate cleanly |
The honest costs
Graphs are not free. Extraction quality varies by domain and needs review. Resolution is a living system that drifts as sources update. Schemas evolve, and every evolution touches the pipeline. The decision rule is simple to state: if relational questions are central to the product, the graph pays for itself; if passage questions dominate, a vector stack with good filters is the better spend, and the graph is a cost you were talked into.
Two domains show where the investment clears the bar. Open-source intelligence is one: the same actor appears under different names and scripts across sources, and resolution with provenance is what makes multi-hop analysis mean anything. Biomedical literature is another, where drug, gene, disease, and trial relationships form chains no single paper states end to end.
Closing
RAG did not fail; it was asked the wrong question shape. Passages answer “what does the corpus say.” Paths answer “how do these things connect.” Build the graph when connection questions are the product, spend the unglamorous effort on entity resolution, and let vectors do what they do best: bring the human the evidence for each hop.