GraphRAG as Scoped Context for Production Agents

The default strategy for giving an AI agent more knowledge is to keep adding context.

Add the conversation history. Add retrieved documents. Add tool definitions. Add the last ten tool outputs. Add the system architecture. Add a memory summary. Add the user’s preferences. Add a few examples so the model behaves consistently.

Every addition is individually defensible. Together, they create a new failure mode: the agent has access to the right information, but the signal is buried inside an increasingly noisy working set.

This is the practical problem behind context rot. A model may support a very large context window, but that does not mean every token in that window receives equal attention or remains equally useful. The agent’s working context accumulates stale facts, duplicated instructions, irrelevant tool output, and evidence from earlier steps that no longer matches the current task.

The answer is not simply a larger window. The answer is a better context architecture.

My preferred pattern is to treat a knowledge graph as a context control plane. The graph is not pasted into the prompt. It is queried before each meaningful agent step to identify the entities, relationships, evidence, permissions, and current state that matter now. The result is a bounded context packet assembled for a specific task.

That is the role I see for GraphRAG in production agents: not unlimited memory, but scoped context reconstruction.

Long context is not the same as durable memory

A context window is working memory. It contains the information available to the model during one inference. Agent memory is a different system problem: deciding what should persist, how it should change, and what should be recalled for a future task.

Conflating the two leads to fragile architectures. If an agent needs to remember a system owner, an approved metric definition, or the result of a prior investigation, putting the entire history into every future prompt is not a memory strategy. It is replay.

Research has repeatedly shown why replay is insufficient.

The Lost in the Middle study found that model performance can vary substantially based on where relevant information appears in a long input. Models often used evidence near the beginning or end more reliably than evidence in the middle. The RULER benchmark extended long-context evaluation beyond simple retrieval into multi-hop tracing and aggregation, showing that advertised window size and effective task performance are not the same thing.

Long-term memory research finds a related problem across sessions. LongMemEval evaluates information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Its results show substantial degradation as assistants work over sustained histories. The newer LongMemEval-V2 frames memory as context gathering from as many as 500 prior trajectories and more than 100 million tokens. The challenge is not to replay that history. It is to retrieve compact evidence that helps the agent behave like an experienced colleague.

Anthropic describes the operational response as context engineering: repeatedly curating the smallest high-signal set of tokens from a much larger universe of possible information.

This distinction is the foundation:

Long context is how much the model can technically receive.
Memory is what the system retains outside the model.
Context engineering is how the system decides what the model receives now.

A knowledge graph can participate in all three, but its highest-value role is in the third.

What context rot looks like in an agent

Context rot is not one defect. It is the combined effect of several kinds of degradation.

Irrelevant accumulation

An agent performs a ten-step investigation. Each tool returns several thousand tokens. By step ten, the prompt contains logs, schemas, intermediate hypotheses, and raw records that were useful earlier but are unrelated to the current decision.

The relevant fact is present, but it competes with everything else.

Stale state

The agent retrieves an owner, status, policy, or user preference that was correct last month. A later event superseded it, but both versions remain in memory. Semantic similarity alone may return the old and new records without a reliable way to identify which one is valid now.

This is especially dangerous because stale context often looks plausible.

Duplicated or conflicting instructions

Long-running agents accumulate summaries, handoffs, tool descriptions, and prior plans. If those artifacts are not versioned and compacted carefully, the model may receive several subtly different representations of the same rule.

Relationship loss

Chunk retrieval finds passages that mention the right terms, but the agent still has to infer how those passages connect. It may retrieve a metric definition, a report description, and a model document without knowing that the model produces the metric and the report consumes it.

Permission leakage

A broad retriever may find semantically relevant content before the system has established whether the current user or agent is authorized to see it. Filtering after retrieval is both risky and inefficient.

Tool overload

Agents connected to many systems may receive hundreds of tool definitions. Even before any business data is retrieved, tool metadata consumes context and creates selection ambiguity. Anthropic has described this problem in the context of large MCP tool surfaces: loading every tool and passing every intermediate result through the model increases cost and slows the agent.

These are not solved by asking the model to “pay attention.” They require the application to control context as a governed resource.

The graph is a map, not the prompt

A knowledge graph represents entities and typed relationships:

Counterparty
  -> belongs_to -> Legal Entity
  -> has_exposure -> PFE Metric

PFE Metric
  -> produced_by -> Risk Engine Run
  -> sourced_from -> Trade Population
  -> appears_in -> Daily Risk Report
  -> governed_by -> Metric Definition

Daily Risk Report
  -> owned_by -> Reporting Team
  -> reviewed_under -> Control

The graph may contain millions of nodes and edges. The model should almost never receive all of it. The graph’s value is that the system can traverse a small neighborhood based on the task.

Microsoft’s GraphRAG project makes a useful distinction between query modes:

Local search starts from specific entities and combines graph structure with related source text.
Global search reasons over community summaries to answer questions about a corpus as a whole.
DRIFT search combines local entity expansion with broader community context.

Those modes were developed for question answering over private corpora, but the underlying idea is broader. An agent should choose a retrieval scope based on the shape of its current step.

If the task is “inspect this counterparty’s PFE movement,” the agent needs a local subgraph around a counterparty, risk run, portfolio, and report. If the task is “identify recurring causes of PFE breaks this quarter,” it may need community-level patterns plus selected local examples. If the task changes during execution, the context should be rebuilt rather than endlessly appended.

The graph remains external. The agent receives a projection.

A scoped context pipeline

I would structure a graph-backed agent step as an explicit pipeline.

1. Resolve the task and candidate entities

Parse the user’s request and the agent’s current plan into:

task type
candidate entities
time range
desired output
risk level
required capabilities

Entity resolution should be conservative. “Exposure” may refer to current exposure, potential future exposure, expected exposure, a dashboard field, or an internal dataset. If the system cannot resolve the term confidently, it should ask a question instead of expanding every possible interpretation.

2. Apply authorization before traversal

Permissions belong in the retrieval plan, not in a cleanup step after retrieval.

The system should determine:

which entity types the user may inspect
which edge types the agent may traverse
which source documents may be retrieved
which tools may be exposed
which actions require approval

This allows the graph to function as an authorization-aware map. A user may be permitted to inspect a published risk metric but not the underlying counterparty records. Another user may access the data but not invoke a remediation action.

3. Choose an expansion policy

Graph traversal must be bounded.

A useful expansion policy might include:

one or two hops from resolved entities
an allowlist of relationship types
a time window
a confidence threshold
a maximum number of nodes per type
a token budget for the final projection

The traversal should be shaped by the task. An impact-analysis agent may follow DEPENDS_ON, PRODUCES, and CONSUMES. A policy agent may follow GOVERNS, IMPLEMENTS, and OWNED_BY. A debugging agent may follow EXECUTED_BY, FAILED_AT, EMITTED, and DEPLOYED_AS.

Unbounded traversal simply recreates context overload in graph form.

4. Retrieve evidence attached to the subgraph

The graph supplies structure, but the model still needs evidence.

Each selected node and edge should point to authoritative source material:

a metric definition
a model run record
a policy paragraph
an API response
a code location
a ticket
a deployment event
a conversation turn

This is where vector or lexical retrieval remains useful. The graph identifies where to search and why. Document retrieval supplies the human-readable evidence needed to explain or verify the relationship.

GraphRAG is strongest when it does not force a choice between graphs and text. The graph narrows the search space; the retriever gathers supporting passages; the agent receives both.

5. Rank facts for the current decision

Not every fact in the subgraph deserves prompt space.

Rank candidates using signals such as:

direct connection to the focal entity
relationship relevance to the task
source authority
recency and temporal validity
user permissions
confidence
contradiction status
prior use in the current run

The goal is context precision, not maximum recall. The packet should contain enough information to complete the step and no more than the step can use reliably.

6. Compile a typed context packet

The context compiler turns graph results into a stable interface for the model.

{
  "task": {
    "type": "explain_metric_change",
    "as_of": "2026-06-13T14:00:00Z"
  },
  "focus": {
    "entity_id": "counterparty:8421",
    "label": "Example Holdings"
  },
  "facts": [
    {
      "subject": "pfe:daily",
      "relation": "produced_by",
      "object": "risk_run:2026-06-13",
      "valid_at": "2026-06-13",
      "source": "risk-run-api",
      "confidence": 1.0
    }
  ],
  "evidence": [
    {
      "source_id": "model-methodology:v7",
      "excerpt": "Potential future exposure is calculated...",
      "supports": ["pfe:daily"]
    }
  ],
  "tools": ["get_trade_changes", "get_market_data_moves"],
  "constraints": [
    "Do not infer causality without supporting evidence.",
    "Escalate if model-run metadata is incomplete."
  ]
}

This is more reliable than prose assembled ad hoc. The model can distinguish governed facts, supporting excerpts, available tools, and behavioral constraints.

7. Execute one step, then refresh

The packet should have a lease.

After a meaningful tool call or state change, the system should decide whether the context is still valid. A trade query may introduce new identifiers. A deployment check may show that the running version differs from the version in the initial graph. A user clarification may change which “exposure” metric is in scope.

Rather than preserving the old packet and appending the new result, the system can:

write the new event to run state,
update or invalidate relevant graph facts,
rerun scoped retrieval,
compile the next packet.

This is how the agent avoids carrying every historical intermediate into every future inference.

Three layers of agent context

It helps to separate context into three layers.

Working context

This is the model’s immediate packet: current task, selected facts, recent tool outputs, available actions, and response requirements. It should be small and disposable.

Episodic run state

This stores what happened during the current workflow:

steps completed
tools called
observations
unresolved hypotheses
approvals
generated artifacts

The agent can retrieve from this layer when needed, but it does not need to replay the full event stream every step.

Durable knowledge

This is the governed graph and its linked evidence:

entities
definitions
relationships
owners
permissions
policies
system topology
valid time
provenance

Durable knowledge should change more slowly than run state, although some domains require frequent updates.

This separation resembles the virtual-memory idea in MemGPT: limited active context can be backed by larger external memory tiers. The graph adds a relational access path over that memory. Instead of paging in the most semantically similar text alone, the system can retrieve the facts connected to the current entity and task.

Why temporal edges matter

An agent memory system must know not only what was true, but when it was true.

Consider these facts:

Application A -> owned_by -> Team Red
Application A -> owned_by -> Team Blue

Without time, the graph contains a contradiction. With validity intervals, it contains a history:

Application A -> owned_by -> Team Red
valid_from: 2024-01-01
valid_to:   2026-03-31

Application A -> owned_by -> Team Blue
valid_from: 2026-04-01
valid_to:   null

The context compiler can now answer “who owns this application?” differently from “who owned it during the March incident?”

This is not a niche concern. Long-term agent memory constantly encounters changing preferences, owners, statuses, policies, plans, and system versions. LongMemEval explicitly includes knowledge updates and temporal reasoning. The 2025 Zep temporal knowledge graph paper explores graph-based agent memory that synthesizes conversational and business data while maintaining historical relationships.

The broader lesson is simple: agent memory needs mutation semantics. A new fact should be able to supersede, invalidate, or qualify an old fact. Otherwise retrieval can make context rot worse by repeatedly resurfacing obsolete information.

How scoped GraphRAG reduces context rot

GraphRAG does not eliminate model limitations. It reduces the amount of irrelevant and ambiguous material those limitations have to operate over.

It replaces accumulation with reconstruction

The system builds context from current state instead of carrying the full prompt forward forever.

It compresses relationships structurally

A path such as:

Metric -> produced_by -> Model -> consumes -> Dataset

can preserve more operational meaning than several pages of documents that mention those components independently.

It makes scope explicit

Traversal policies define which entities and relationships belong to the current task. This is easier to inspect and test than an opaque similarity search across the entire corpus.

It supports temporal filtering

Validity intervals and event timestamps let the system prefer facts that are current for the requested time.

It preserves provenance

Facts and edges can point back to source records. The model can explain which path it followed and which evidence supports each claim.

It narrows tools as well as data

The graph can connect workflows and entity types to relevant tools. A counterparty investigation may expose risk and trade tools without loading deployment, HR, or content-management tools.

It creates natural checkpoints

Each context packet can be logged, evaluated, and compared with the final action. This makes retrieval failures observable.

The key is not that graphs are inherently immune to noise. The key is that graph traversal gives the application a programmable way to control noise.

A risk investigation example

Suppose an analyst asks:

Why did Potential Future Exposure increase for this counterparty today?

A weak implementation retrieves every document containing “PFE” and sends recent conversation history plus a large schema dump to the model.

A scoped implementation proceeds differently.

First, it resolves the counterparty and the approved PFE metric. It identifies today’s and yesterday’s calculation runs. It checks the user’s entitlements before traversing into trade-level data.

Then it builds a local subgraph:

Counterparty
  -> has_portfolio -> Trade Population
  -> measured_by -> PFE Metric

PFE Metric
  -> produced_by -> Today's Risk Run
  -> produced_by -> Yesterday's Risk Run
  -> defined_by -> Approved Methodology
  -> appears_in -> Daily Counterparty Report

Today's Risk Run
  -> used_market_data -> Market Snapshot
  -> used_model -> Model Version
  -> emitted_issue -> Data Quality Alert

The agent retrieves only the evidence linked to those nodes: changed trades, market moves, model-run metadata, the applicable methodology excerpt, and the active data-quality alert.

The final context packet may be a few thousand high-value tokens rather than tens of thousands of broadly relevant tokens. The model can form hypotheses, call targeted comparison tools, and distinguish observed causes from unresolved possibilities.

If the agent discovers that today’s run used a new model version, the context is rebuilt around that version and its release record. If the data-quality alert invalidates one source feed, the system marks affected evidence accordingly. If the user lacks trade-level permission, the agent returns an aggregate explanation and states the access limitation.

That is GraphRAG as agent infrastructure, not chatbot decoration.

What the graph does not solve

GraphRAG creates new engineering responsibilities.

Bad extraction creates bad structure

If an LLM invents an entity or relationship during graph construction, the error may look more authoritative because it is now represented as a typed edge. High-impact graphs need source linkage, confidence, validation, and ownership.

Entity resolution is difficult

Organizations contain aliases, renamed systems, duplicated identifiers, subsidiaries, and overloaded terminology. Incorrect merges can contaminate every downstream traversal. Incorrect splits can hide relevant connections.

Ontologies can become political

The hard part is often deciding what a concept means, who owns it, and which relationship is authoritative. A graph database does not resolve organizational disagreement.

Traversal can still over-retrieve

A highly connected node can explode into thousands of neighbors. Every query mode needs budgets, edge allowlists, relevance ranking, and stopping conditions.

Permission models must propagate

Security cannot stop at node visibility. Derived paths, community summaries, cached context packets, and model outputs can all leak restricted facts if permissions are not enforced end to end.

Temporal truth needs maintenance

Edges require update and invalidation paths. A graph that only adds facts will eventually contain a large amount of stale truth.

Global GraphRAG can be expensive

Microsoft’s original GraphRAG research constructs entity graphs and community summaries to answer broad corpus-level questions. That indexing and summarization can be costly. LazyGraphRAG explores a different quality-cost tradeoff that avoids up-front summarization. The right architecture depends on query patterns and corpus stability.

The graph should earn its maintenance cost by powering more than prompt augmentation: lineage, impact analysis, ownership, governance, search, workflow routing, and audit.

How I would evaluate the system

Final-answer quality is not enough. A scoped context system should be evaluated at every boundary.

Graph quality

entity-resolution precision and recall
relation extraction accuracy
duplicate entity rate
stale or invalid edge rate
percentage of facts with authoritative provenance

Retrieval quality

focal entity resolution accuracy
relevant path recall
context precision
temporal correctness
permission-filter correctness
evidence coverage

Context efficiency

input tokens per successful task
irrelevant-token rate
duplicate fact rate
number of context refreshes
tool definitions exposed per step

Agent behavior

factual grounding
path and source citation accuracy
correct tool selection
abstention when evidence is insufficient
correct handling of conflicting or superseded facts
human approval and rework rates

Operational behavior

graph freshness
retrieval latency
context compilation latency
indexing and update cost
audit completeness

The evaluation set should include ambiguous names, permission boundaries, stale facts, conflicting evidence, missing relationships, and graph extraction errors. If every test assumes a clean graph, it does not test a production graph.

A practical implementation path

I would not begin by modeling the entire enterprise.

Start with one workflow where relationships already determine the answer. Good examples include:

risk metric investigation
data lineage and report impact
application ownership
model inventory
incident analysis
policy-to-control mapping
agent tool routing

Define a small ontology around that workflow. Connect every important edge to evidence. Add validity time and ownership from the beginning. Implement local traversal first, because entity-centered questions are easier to evaluate than broad corpus synthesis.

Then build a context compiler with a strict token budget. Separate graph facts from document evidence and tool results. Log the exact packet used for every inference. Measure whether the scoped packet performs better than vector retrieval alone and full-history prompting.

Only after local retrieval is reliable should the system add community summaries, global search, agent-written memory, or automatic graph evolution.

The goal is not to build the largest graph. It is to build the smallest graph that lets the agent gather the right context reliably.

The design principle

An agent should not remember by carrying everything forward.

It should externalize durable state, retrieve what the current step requires, and reconstruct a bounded working context from governed sources. A knowledge graph is useful because it represents the relationships that determine relevance: which facts belong together, which source is authoritative, which version is current, which user may see it, and which tool can act on it.

That does not make the model immune to context rot. It changes the application from a passive prompt builder into an active context manager.

The production pattern I expect to matter is:

large external memory
  -> entity resolution
  -> permission-aware graph traversal
  -> evidence retrieval
  -> context ranking and budgeting
  -> small typed context packet
  -> one agent step
  -> state update
  -> reconstruct again

The graph is not the answer, and it is not the entire memory.

It is the map the system uses to decide what the agent should know next.