Stateful Agent Checkpointing and Mid-Run Recovery

Long-running AI agents don't fail because the models get dumb halfway through a task. They fail because most agent pipelines still treat a run as something that either finishes or vanishes, with nothing in between. The agentic AI market is estimated at $9.9 billion in 2026, growing more than 40% a year, and Gartner predicts that 40% of agentic AI projects will be canceled by the end of 2027, even as the share of enterprise applications embedding task-specific agents was still under 5% in 2025. Yet an IDC/AWS study found less than 7% of organizations have made it to full production with even one agentic use case. Gartner separately projects that 40% of agentic AI projects will be canceled by the end of 2027, and the reason isn't that the models can't reason. It's that the systems around them can't survive a bad afternoon.

Picture the naive setup: one process, an LLM loop, some tool calls, a return value at the end. Fine for a task that finishes in 20 seconds. Now stretch that same task to four hours, and an API call at step 4 of 7 returns a 503. The whole workflow restarts from zero. Compute already spent is gone, side effects from earlier steps might fire twice, and nobody watching the system has any idea what actually happened before it died. Agents that run past four hours without state persistence carry a 90% higher risk of total task failure from timeouts or infrastructure hiccups alone. This is a distributed-systems problem wearing an AI costume, and the fixes look almost exactly like the fixes distributed systems engineers have been using for decades: checkpoint the state, detect the failure fast, recover from somewhere other than the start line.

What state an agent actually accumulates during a run

Ask what an agent "remembers" mid-task and most people picture a chat log. That's only half of it, and treating it as the whole picture is exactly why so many recovery attempts come back broken.

There are two separate layers of state building up while an agent runs:

• Application-level state: the conversation history, results already returned by tools, the reasoning chain, and intermediate artifacts like draft plans, generated code, or partial proofs.
• OS-level state: filesystem changes, packages the agent installed, background processes it spawned, anything sitting inside a container or microVM that isn't visible from the chat transcript.

Framework-level persistence commonly captures only the first layer. It saves the conversation and the tool outputs, then calls it done. But an agent that compiled a dependency, kicked off a background service, and rewrote three files mid-run can't just pick back up from its chat history. The files are still half-written. The service may or may not still be running. None of that shows up in a transcript.

There's also a third dimension, mostly relevant in regulated environments: a run history kept for audit purposes, which serves a different function from whatever slice of state the agent actually needs to resume. Audit and resume are different jobs, even though people often bolt them onto the same log.

This is the taxonomy that a checkpoint schema has to respect. Miss the OS layer, and recovery looks successful right up until the agent tries to use a file that no longer exists.

What a checkpoint is and what it must contain to be useful

A checkpoint, in the terms LangGraph and LangChain use in their documentation, is a snapshot of an agent's complete state at a specific point in execution, tagged with a unique, monotonically increasing ID. Simple definition. The hard part is deciding what "complete" actually means.

At minimum, a checkpoint needs:

• Full conversation history up to that point
• Every tool call result already completed
• The reasoning chain and any intermediate artifacts
• The current position in the workflow graph: which step, which branch

That's the floor. A production-grade checkpoint goes further and adds OS-visible state (filesystem layer, running processes, installed packages) plus serialization metadata: which serializer wrote it, what schema version it used, and when.

Granularity is a real design decision, not a default to accept blindly. Checkpoint at the phase level and writes stay cheap, but a failure means redoing a bigger chunk of work. Checkpoint at the node level, on every single tool call, and a 50-step workflow produces 50 persisted states. Recovery becomes almost free, but storage volume climbs fast. The right answer depends on how expensive each step is to redo and how reversible it is. A step that calls a paid API three times shouldn't share a checkpoint boundary with a step that just formats text.

None of this matters, though, without idempotency. The principle is well established: checkpointing without idempotency guarantees produces corrupted data or duplicated side effects the moment something resumes. Saving state is only half the contract. The other half is making sure that replaying an action after a crash doesn't charge a customer's card twice or send the same email again.

One underrated bonus of doing this right: time-travel debugging. Because state is persisted at each step, developers gain a record they can use to trace back through a run. For debugging the kind of nondeterministic weirdness LLMs produce, where the same prompt gives a different answer on the retry, that's not a nice-to-have. It's often the only way to actually reproduce the bug.

The three mechanisms that form a complete fault-tolerance loop

Checkpointing by itself is not fault tolerance. It's state preservation, and preserving state does nothing if nobody notices the agent died and nothing ever picks the saved state back up.

Fault tolerance needs three mechanisms working together:

• Heartbeat monitoring: catches the failure quickly and triggers recovery.
• Checkpointing: saves state at the right granularity so recovery has an actual place to resume from.
• Redundancy, via shadow agents: takes over the work without the user ever noticing a handoff happened.

U.S. Patent 12475151, covering fault-tolerant multi-agent generative AI, describes exactly this pattern: a Shadow Agent tracks the Primary Agent's state so that when the primary fails, the shadow can step in with current state already in hand. When the primary dies, the shadow steps in already caught up, and the transition doesn't surface to whoever's waiting on the output.

Detect the problem, save the scene, switch the instance. That's the whole loop, and it's the same loop that's kept database replication and cloud failover working for years. Agents didn't need a new theory of reliability. They needed someone to bother applying the old one.

Two more distributed-systems patterns have to travel alongside checkpointing or the loop breaks under load. Idempotency, again, so a resumed action doesn't fire twice. And exponential backoff with full jitter, so that when many agents fail at once (a shared dependency going down, say) they don't all slam back into the same service in the same instant and take it down again.

There's also a lifecycle dimension the research hasn't caught up to yet. The Always-On Agents survey (arXiv 2606.30306, June 2026) describes state moving through stages: updated, forgotten, audited, and sometimes rolled back. Call it the return arc, since most of the literature spends its attention on how state gets accumulated and retrieved, and comparatively little on how it gets governed, recovered, or let go of. That's exactly the part production systems tend to skip, and exactly the part that bites teams once something actually goes wrong at 2 a.m.

How LangGraph implements checkpointing and what each backend trades away

LangGraph is the closest thing agent checkpointing has to a reference implementation right now. Its GitHub repo has passed 30,000 stars, and it's among the most actively used agent frameworks going into 2026.

LangGraph splits persistence into two systems that solve different problems. Checkpointers persist a thread's graph state as a series of checkpoints. This is short-term, thread-scoped memory, and it's what powers conversation continuity, human-in-the-loop pauses, time travel, and fault recovery. Stores persist application-defined data that lives outside the graph state entirely, long-term and shared across threads: user preferences, learned facts, anything that should survive past a single conversation.

The choice of checkpointer backend comes with real trade-offs, not cosmetic ones:

• MemorySaver / InMemorySaver: lives in RAM, disappears the second the process restarts. Fine for prototyping. Never put this in production.
• PostgresSaver / AsyncPostgresSaver: the recommended production default. Durable, supports pause and resume, lets you inspect state after the fact.
• SqliteSaver: works, but write performance chokes under high concurrency. Not built for scale.
• DynamoDBSaver: serverless and scales automatically, storing lightweight checkpoint metadata in DynamoDB while pushing larger payloads to S3. AWS has published production guidance specifically for this connector.
• RedisSaver / AsyncRedisSaver: sub-millisecond latency for thread-level persistence, through the langgraph-checkpoint-redis package. Pair it with RedisStore or AsyncRedisStore for cross-thread memory.

Every one of these savers relies on a serialization protocol underneath. The default, JsonPlusSerializer, uses ormsgpack with a fallback to extended JSON handling for LangChain and LangGraph types, datetimes, and enums. An EncryptedSerializer is available when the payload needs to stay confidential at rest.

The decision rule is fairly clean once you frame it by priority. Pick Postgres when correctness and inspectability matter most. Pick Redis when the agent's responsiveness is the thing users will actually notice. Pick DynamoDB when the stack is already AWS-native and operational simplicity is worth more than squeezing out extra milliseconds.

Checkpointing in Microsoft Agent Framework and Amazon Bedrock AgentCore

Microsoft's Agent Framework takes a more automatic approach. It checkpoints at superstep boundaries through a CheckpointStorage protocol in Python and a CheckpointManager class in.NET, through a CheckpointStorage protocol in Python and a CheckpointManager class in.NET. Sequential, multi-step workflows are a natural fit for this approach, since they offer many opportunities for one step to fail and demonstrate the value of automatic recovery.

There's also a memory layer built for retrieval rather than raw recovery: AgentCoreMemoryStore, which saves conversational messages and searches long-term memory within a dedicated memory store. That's a different problem than checkpoint-and-resume, closer to giving an agent a durable long-term memory than to protecting an in-flight task.

Amazon Bedrock AgentCore takes the same core idea and applies it at the session level. Sessions get written to persistent storage inside the service itself, so a container exiting, an in-memory cache disappearing, or the agent application crashing outright doesn't wipe the session's state, according to AWS's own AgentCore documentation.

Line all three frameworks up side by side and the pattern repeats: the checkpoint is the unit of recovery everywhere. What changes between them is where that checkpoint lives, how it gets queried back out, and how long it sticks around before it's cleaned up.

The semantic gap between application-level and OS-level recovery, and how Crab addresses it

Here's a wrinkle that complicates the tidy story above. Most of an agent's turns don't touch OS-level state at all, so most OS-level checkpointing is wasted effort. Research out of HKUST (the Crab paper, arXiv 2604.28138, April 2026) found that over 75% of agent turns produce no recovery-relevant OS state whatsoever. But the turns that do matter are exactly the ones invisible to application-level frameworks, and getting them wrong is expensive: on shell-intensive and code-repair workloads, chat-only recovery achieves just 8% correctness, according to the same paper.

Eight percent. That's not a rounding error, that's a system that's basically failing every time it needs to actually recover from something that touched the filesystem.

Crab's fix works as a transparent host-side runtime that doesn't require rewriting the agent or swapping out the checkpoint/restore backend underneath it. An eBPF-based inspector watches each turn and classifies its OS-visible effects to decide how much checkpoint granularity that turn actually needs. A coordinator lines checkpoints up with turn boundaries and overlaps the checkpoint/restore work with the time the agent is already sitting idle waiting on the LLM, effectively hiding the cost inside dead time that would otherwise go to waste. A host-scoped engine then schedules checkpoint traffic across sandboxes running on the same machine, so they're not all fighting for disk I/O at once.

The results, on shell-intensive and code-repair workloads: recovery correctness climbs from that 8% chat-only baseline to 100%, checkpoint traffic drops by as much as 87%, and total execution time stays within 1.9% of a run with no faults at all.

For anyone building agents, the practical line is fairly clean. An agent that only reads web pages and calls APIs can probably get by on framework-level checkpointing alone. An agent that runs shell commands, installs packages, or edits files on disk needs OS-level awareness, or its recovery is going to look a lot like that 8% number.

Delta-based checkpointing for agents that need to checkpoint at high frequency

Full-state checkpointing is correct. It's also slow, and slow becomes a real problem the moment an agent needs to checkpoint hundreds of times in a single run rather than a handful.

The existing tools for full checkpoint/restore aren't built for that frequency. Docker commits take several seconds once there's any nontrivial layer change to capture. VM-level snapshotting runs at a pace that becomes a real bottleneck at high checkpoint frequencnds. Multiply either of those by a few hundred checkpoints and the overhead starts to dominate the run.

DeltaBox, out of Shanghai Jiao Tong University and Huawei (arXiv 2605.22781, May/June 2026), starts from a fairly simple observation: consecutive sandbox states between two checkpoints usually differ by very little. So instead of duplicating the whole state every time, duplicate just the delta. Two components do the work. DeltaFS checkpoints file state by freezing the current writable layer and inserting a fresh one on top, so every update afterward becomes copy-on-write and rolling back is just switching which layer is active. DeltaCR handles process state the same way, rolling back by forking from a frozen template process backed by a CRIU dump.

On SWE-bench and reinforcement-learning micro-benchmarks, DeltaBox checkpoints in roughly 10.83 milliseconds, fast enough to hide entirely under the time the agent already spends waiting on the LLM, and rolls back in about 1.86 milliseconds.

That kind of speed changes what's actually possible under a fixed time budget. Tree-search and RL rollout workloads that used to be bottlenecked by checkpoint overhead, where every extra branch explored meant another slow snapshot, suddenly get to explore a lot more of the tree for the same cost. Most production teams building a customer-facing agent don't need sub-10-millisecond checkpointing today. Teams doing test-time compute, RL post-training, or agentic tree search very much do, and for them this isn't a nice performance bump, it's the difference between a search that's viable and one that isn't.

Checkpoint-aware recovery at the reasoning level: REPOT's verified-prefix approach

Everything above deals with infrastructure failing: a crash, a timeout, a dead container. There's a separate failure mode that has nothing to do with infrastructure at all. The agent's plan is completely sound for the first several steps, and then the next step turns out to be illegal, invalid, or impossible to execute. Standard practice throws out the whole trajectory, verified prefix included, and starts the planning over from scratch.

REPOT, short for Recoverable Program-of-Thought, out of UC Santa Cruz (arXiv 2605.30052), takes a narrower and cheaper approach than jumping straight to full tree search. It works in three steps. First, run Program-of-Thought planning once, the normal way: emit a Python program, execute it, parse out the resulting list of moves. Second, verified replay: walk through the proposed actions one at a time against the actual environment, and keep accumulating a "verified prefix" of moves that actually worked, right up until the first one fails. Third, check whether that verified prefix already reaches the goal. If it does, done. If it doesn't, issue exactly one repair call to the LLM, and give it everything it needs to fix the ending rather than the whole plan: the verified prefix, the verified state at the point where things broke, and the verifier's error message describing what went wrong.

The savings come from not discarding what already worked. A plan that's correct for 40 steps and breaks on step 41 doesn't need 41 steps of fresh reasoning, it needs one targeted patch at the point of failure. That's the same instinct behind checkpointing at the infrastructure level, just moved up a layer: don't rebuild what you already know is good, recover from the last point you can actually trust.