Context Window Management in Long Agentic Workflows

Context window exhaustion happens even when the window is huge. That's the core problem in long agentic workflows: an agent running dozens of tool calls per task can burn through its entire context budget before the task is done, no matter how big the window starts out.

Research on enterprise ERP workflows found exactly this. Multi-step agents doing real work, filling out forms, navigating portals, chaining tool calls, ran out of room mid-task. Each tool call can pull in roughly 11,000 tokens just from loading a file. Running a dozen of those in a document-heavy or search-heavy workflow turns the math against you fast. Even at 128,000 tokens, agentic RAG systems studied in the literature hit their ceiling often enough to cut reasoning short before the agent finishes the job.

The model usually has no idea how full its own window is, which makes this worse than a simple capacity problem. VISTA research out of CUHK and Tencent (Xu et al.) calls this proprioceptive blindness. Frontier models, given just the prompt, cannot reliably tell you their own block size, what's recent, or how much budget is left. The model isn't failing to plan around its limits, it literally can't see them.

So bigger windows push the exhaustion point further out, but they don't fix the shape of the problem. Active management is still required, and the field has landed on four broad families for doing it: truncation, recurrent compression, summarization, and structural redistribution (external memory or splitting work across agents). The families differ on two axes: who decides what gets cut (a fixed rule, a separate system, the agent, or a trained policy), and whether the cut is lossless or lossy. Summarization is lossy by design. Archiving with recovery, as in VISTA, can be lossless. Every strategy below sits somewhere on that grid.

Start with the cheapest option: sliding window or recency pruning. Keep the last N tool call/response pairs, throw out everything older. It works because in enterprise tool-use tasks, old tool responses are full of metadata, form state, and navigation breadcrumbs that stop mattering once the agent moves on.

It fails on long-range dependencies. If a file path, a portal URL, or a constraint set 30 steps back needs to resurface now, pruning drops it silently, with no flag and no warning. The VISTA paper gives a clean example: auto-compact drops an exact portal URL from the block, and the agent sends a broken link. It never knew the block was gone. Recency pruning is a solid default for repetitive, locally structured tasks. It gets worse the more a task depends on sparse evidence from far back in the trajectory.

Summarization as compression: what it preserves, what it erases, and a counterintuitive finding

Summarization sits one rung up from pruning: instead of just discarding old content, you compress it into a shorter form and keep the gist. One study found that layering automated summarization on top of pruning hit 91.6% complete itemization, the best result in the study, while token use stayed close to pruning alone at 553,374 tokens. On structured, enterprise-style tasks, summarization plus pruning beat pruning by itself by a wide margin.

But a JetBrains Research finding on 250-turn agent trajectories complicates the story. Observation masking, a cruder, cheaper method than LLM-based summarization, solved tasks at a 2.6% higher rate while costing 52% less. The reason gets at something summarization does quietly and badly: LLM summaries smoothed over failure signals, making a failed attempt read like partial progress. That stretched agent trajectories by 13 to 15%, because agents kept retrying paths they'd already tried and already failed.

Putting the two findings side by side reveals a real tension. Summarization helps on structured enterprise tasks where completeness is the goal. It hurts on open-ended, multi-step tasks where the agent needs to know that something didn't work. The right call depends on what kind of task is running.

Discard-75% drops the first three-quarters of tool call history. Crude, but predictable. The broader lesson across all of these: summarization needs guardrails so it doesn't sand down failure evidence into something that looks like progress, and it works best when it's conditioned on how much budget is actually left, not applied on a fixed schedule regardless of need.

That's where token budgeting comes in: deciding, before each call, how many tokens go to system instructions, retrieved documents, conversation history, and tool outputs, then enforcing that split when the prompt gets assembled. A workable partition from agentic systems research looks something like this: 2,000 tokens for the system prompt (capabilities, safety rules, output format), 4,000 for user intent and conversation, a sliding window of the last 8 to 12 tool actions and observations for tool history (older ones get summarized), and a hard ceiling around 80,000 tokens total.

Most compression methods skip this step. They compress the same way regardless of how much room is left, which creates two failure modes depending on which side of the budget you're on. With a loose budget, static compression over-trims and erases evidence that later turns out to matter. With a tight budget, it under-compresses and overflows the ceiling, causing truncation or brittle failure that's hard to debug after the fact.

Agentic RAG systems handle this with threshold enforcement: track token use against a 128,000-token ceiling, throw an internal warning at 90% of budget, force summarization once the threshold hits, and scan tool messages to strip anything not tied to a reference ID worth keeping. Given that a single file fetch can burn 11,000 tokens, that ceiling arrives faster than most teams expect, sometimes in far fewer calls than the budget would suggest.

Budget partitioning matters because it makes eviction rules enforceable. You can't decide what to evict from a zone if you never defined what that zone is allowed to cost.

Even with a good budget in place, agents still make bad keep-or-archive calls, because they can't see block size, recency, archive status, or remaining budget from the prompt alone. That's the same proprioceptive blindness problem from the VISTA paper, and it's what VISTA is built to fix.

VISTA (Visible Internal State for Tool Agents) is a training-free, model-agnostic layer that represents working memory as typed, addressable blocks and gives the agent a runtime dashboard: token usage, recency, archive status, and remaining budget, all visible per block. Archived blocks stay recoverable at full fidelity, which makes this lossless in a way summarization structurally can't be. On LOCA-Bench, VISTA lifted Gemini-3-Flash from 22.7% to 50.7%. On BrowseComp-Plus, it reached 58.0%.

Self-GC, from Xiaohongshu (2026), takes a different route to a similar goal. It turns user turns, tool spans, and skill state into indexed objects, then uses a side-channel planner to propose folding, masking, or pruning each one. The execution harness enforces recoverable sidecars and cache-aware commits, so nothing gets destroyed without a way back. On a 33-session Hard Set, Self-GC pruned 43.95% of prefix tokens while leaving 84.85% of future continuations untouched. In production, split by account at the daytime level, it cut average input tokens by 10 to 15%, with peak reductions near 20%.

Compare that against heuristic masking (chronological pruning, blind tool-output masking): Self-GC's production no-impact rate ran 91.27 to 94.58%, against 77.71 to 87.46% for the heuristic baselines. That gap is the cost of pruning blind versus pruning with some sense of what the future actually needs.

External memory and multi-agent distribution as structural alternatives to compression

Every method above compresses what's already inside the window. The other option is architectural: move knowledge outside the window entirely, and pull it back in only when it's needed, or split the trajectory across multiple agents so no single one ever holds the full history.

External memory, grounded in retrieval, works by leaving the base model untouched and pulling in only the most relevant slice of content at each step. The agent never needs the full document history sitting in its window; it just needs whatever's relevant to the current step. Retrieval ranking can miss something that turns out to matter, and how fresh the retrieved content is depends entirely on how often the underlying index gets updated.

Multi-agent distribution takes a different cut at the same problem. EXTAGENTS, from Tsinghua and Alibaba, splits knowledge processing across Seeking Agents and a Reasoning Agent, working in parallel through global knowledge synchronization, instead of piling every fact onto one agent's growing trajectory. The paper names two failure points in naive multi-agent setups that this design targets directly: knowledge that gets lost across agent boundaries, and orchestration overhead that eats the savings the approach was meant to deliver.

A synthesis from Zylos AI, drawing across methods rather than a single controlled study, found that combining memory patterns like these could cut token costs by 80 to 90% while lifting response quality by 26%. This is a directional signal from combining approaches, not a single benchmark result to hang everything on.

One practical detail belongs here too, even though it sits a level below architecture: when an agent pulls in web content, there's a real difference between markdown stripped down to the main content and raw markdown carrying the full page, boilerplate included. For anything an LLM is going to read, especially at per-token cost, the stripped version is almost always the one worth paying for. That fits a broader two-stage pattern visible across these systems: one stage fetches and cleans the raw material, handling the messy parts, rendering, anti-bot defenses, proxy rotation, and a second stage hands the LLM something already structured, so the model spends its reasoning budget on the task instead of on parsing.