GraphRAG and the Global Question Problem

Some questions have no answer to retrieve

Every technique in this module so far answers the same kind of question: the answer exists in some passage, go find it. Chunking decides whether that passage is findable, parsing decides whether it survived ingestion, query understanding decides whether the search key reaches it, and late interaction decides how precisely it is scored. All of it assumes the answer is in there somewhere.

A whole class of real question breaks that assumption, and it breaks it structurally rather than by degree.

three questions asked of one corpus: 3,000 incident postmortems

Q1  "what caused the March 14 checkout outage"
    is the answer inside some chunk?  YES, one postmortem contains it
    a top-8 retrieval can return it   LOCAL

Q2  "which services has the schema registry taken down"
    is the answer inside some chunk?  PARTLY, spread across ~20 reports
    a top-8 retrieval returns a slice LOCAL, at the edge

Q3  "what are the recurring failure themes across these reports"
    is the answer inside some chunk?  NO
                                      no postmortem states a theme.
                                      the answer is a property of the
                                      set, not of any member of it
    a top-8 retrieval cannot return it   GLOBAL

Why top-k cannot reach a global answer

Retrieval ranks by similarity to the query. For a local question that is exactly right: the passage that answers "what caused the March 14 outage" is the passage that most resembles that question. For a global question the ranking selects on the wrong property. "Recurring failure themes" resembles documents that discuss themes, retrospectives, and postmortem process, and the actual evidence for a theme is three hundred ordinary incident reports that each describe one instance and never name the pattern.

This is not a k problem, a chunking problem, or a reranking problem. It is a mismatch between what retrieval optimizes (resemblance to the query) and what the question requires (coverage of the corpus). No amount of the first buys the second, which is why the answer is a different index rather than a better one.

The GraphRAG index

The construction is four stages, and each one is an ordinary thing you already know applied to text.

1. Extract. An LLM reads each chunk and emits entities and the relations between them. "The schema registry rejected a malformed Avro record, which stalled the ingest pipeline" becomes nodes and a typed edge.
2. Build. Merge those across the corpus into one graph, so an entity mentioned in forty reports is one node with forty pieces of evidence attached.
3. Partition. Run Leiden community detection, recursively: detect communities, then detect sub-communities inside each, down to leaves that cannot be partitioned further. Every level of the resulting hierarchy is a partition of the graph that is mutually exclusive and collectively exhaustive, which is the property that makes divide-and-conquer summarization sound.
4. Summarize. Generate a summary for every community at every level. A leaf community summarizes its entities and relations; a parent summarizes its children rather than the raw text, so the hierarchy is summaries of summaries.

entities and relations from 3,000 incident reports, partitioned by
Leiden, recursively

  level 0, root communities
     C0  "payments platform"          C1  "data platform"
      |                                |
  level 1, sub-communities             |
     C0.0 "card authorization"        C1.0 "ingest pipeline"
     C0.1 "settlement batch"          C1.1 "warehouse queries"
     C0.2 "fraud scoring"             C1.2 "schema registry"

  every node carries a generated summary. C0's summary is written from
  the summaries of C0.0, C0.1 and C0.2, not from the source chunks, so
  reading the level-0 row is reading the whole corpus at one resolution.

  choosing a level chooses a resolution: level 0 is a handful of broad
  summaries, level 1 is more of them and more specific, and the leaves
  are close to the reports themselves.

Two query modes over one index

Global search is a map-reduce. Pick a community level, hand every summary at that level to the model in parallel with the question, collect the partial answers, and reduce them into one. Nothing is retrieved by similarity, because every community at that level participates. That is what buys coverage.

Local search starts from the entities the question mentions, walks their neighborhood in the graph, and pulls the connected entities, relations and source chunks. That is the mode for "what caused the March 14 outage" if you route a local question here at all, and for most systems you would not.

The published evaluation makes the efficiency argument concretely: answering with root-level community summaries took 26,657 context tokens on one dataset against 1,014,611 for summarizing the source texts directly, roughly 2.6 percent of the context, because the hierarchy has already compressed the corpus once at indexing time.

The cost cliff

Which is where the bill arrives. Look at what stage 1 and stage 4 above actually are: an LLM call per chunk, plus an LLM call per community.

corpus: 3,000 incident reports x 4,000 tokens = 12,000,000 tokens
chunked at 600 tokens                          = 20,000 chunks

rates used in this example (stated here, not quoted from a vendor):
  $0.25 per million input tokens, $1.25 per million output tokens
  $0.02 per million tokens embedded

entity and relation extraction, one call per chunk
  input   600 chunk + 400 instruction = 1,000 tokens
  20,000 x 1,000 = 20,000,000 input   -> 20 x $0.25 = $5.00
  20,000 x 500   = 10,000,000 output  -> 10 x $1.25 = $12.50

community summarization, say 1,400 communities across all levels
  1,400 x 3,000 =  4,200,000 input    -> 4.2 x $0.25 = $1.05
  1,400 x 400   =    560,000 output   -> 0.56 x $1.25 = $0.70

  full GraphRAG indexing                        = $19.25

the same corpus, embeddings only, for a vector index
  12,000,000 tokens x $0.02 per million         = $0.24

  19.25 / 0.24 = about 80x the indexing cost of the vector pipeline

Two things about that number. It is small here because the corpus is small, and it is linear in corpus size, so the same 80x on 300,000 reports is roughly $1,925 against $24, and on a corpus that is re-indexed whenever documents change it is a recurring bill rather than a one-time one. And it is a multiplier on the stage that vector RAG made almost free, which is why teams pilot GraphRAG successfully and then fail to fund it.

LazyGraphRAG: defer the calls

The lever is that stage 1 and stage 4 are the only expensive stages, and neither is needed until a query asks. LazyGraphRAG builds the graph without an LLM at all, using noun-phrase extraction to pull out concepts and their co-occurrences, then runs the same community detection over that cheap graph. No summaries are generated at indexing time. When a global query arrives, the LLM work happens then: refining the query, judging relevance, and generating the answer over the communities that matter for that question.

Microsoft reports the indexing cost of this design as identical to vector RAG, and 0.1 percent of the cost of full GraphRAG. Microsoft's 1000x gap assumes far more expensive extraction than the rates above; on these rates the same design lands nearer 80x. The trade is exactly what the name says: query cost rises, and it rises only for the global queries that need it, while local traffic never touches the graph at all.

Table

The first and last rows are the interesting pair: the same indexing cost, and a capability gap between them. Everything in the middle is either paying at indexing for what the last row defers, or trying to reach the last column with a knob that cannot get there.

Design	LLM calls at indexing	LLM calls at query	Answers global questions
Vector RAG	none	one generation per query	no, structurally
Vector RAG, large k	none	one generation over more chunks	no, the sample is still similarity-ranked
Full GraphRAG	one per chunk, one per community	map over community summaries, then reduce	yes
LazyGraphRAG	none, noun phrases and graph statistics	query refinement, relevance judging, generation	yes

Interview nuance: the strong answer is a router, not a religion. In almost every product, the overwhelming majority of traffic is local and belongs on the hybrid pipeline the RAG architecture lesson already built, which is cheaper, faster and better at it. The graph exists for the minority of questions that are global, so the design question an interviewer is actually asking is how a query gets classified into the right path, and what happens when the classifier is wrong. A misrouted local question sent to global search is slow and expensive; a misrouted global question sent to top-k returns a confident answer built from eight documents out of three thousand, which is the worse failure because it looks like an answer.

Recap: a global question is one whose answer is a property of the corpus rather than of any passage in it, so similarity ranking selects on the wrong property and no k fixes it. GraphRAG answers it by extracting an entity graph, partitioning it with recursive Leiden community detection into a hierarchy of mutually exclusive levels, summarizing every community, and map-reducing over the summaries at a chosen level. That costs an LLM call per chunk and per community, which lands around 80x vector-RAG indexing on the stated example. LazyGraphRAG builds the graph from noun-phrase co-occurrence and defers every LLM call to query time, reported at vector-RAG indexing cost. The shippable design routes local traffic to the existing pipeline and reserves the graph for the questions that need coverage.

Sources: From local to global: a graph RAG approach · LazyGraphRAG · Microsoft GraphRAG · Leiden community detection