Long-Horizon Agents, Memory, Compaction, Context Discipline
Pull-quote: “A long-horizon agent that keeps its state in the transcript is a process that keeps its database in RAM. It works until the first restart, and there is always a restart.”
Why this matters
Agents that run for minutes are a prompting exercise. Agents that run for hours, across hundreds of tool calls, are a systems exercise, and the system in question is memory. The context window is finite, attention over it degrades before it fills, and a transcript that accumulates every observation eventually drowns the signal in its own history. Long-horizon reliability is not achieved by larger windows; it is achieved by context discipline.
The window is a budget
Treat context like a memory budget with an allocation policy, because that is what it is. Every token in the window competes for attention with every other token, so a stale error message occupies capacity that the current subtask needs. The failures this produces are recognizable: instructions from the start of the run silently lose force, the agent re-reads files it already processed, and earlier mistakes echo into later reasoning because they were never evicted. The allocation policy that works keeps standing instructions and the goal always resident, keeps the active subtask’s working set fresh, and aggressively evicts what is superseded: raw dumps that were already summarized, tool results that later results replaced, dead ends the plan abandoned.
Compaction
Compaction is checkpointing for context: periodically replace the long transcript with a structured summary and continue with the summary plus recent turns. The design question is what the summary must preserve, and the answer is state, not narrative: the goal and its constraints, decisions made with one line of rationale, artifacts produced (paths, IDs, URIs), what was verified, and what remains. What compaction should shed is process: the retries, the dumps, the abandoned branches. A useful acceptance test: could a fresh agent resume the task from the summary alone? If not, the summary is a recap, not a checkpoint.
Episodic vs semantic memory
| Memory type | What it holds | Where it lives | Retrieved by |
|---|---|---|---|
| Working | Active subtask, recent turns | Context window | Always resident |
| Episodic | What happened in this run or past runs | Logs, run records, transcripts | Recency, run ID |
| Semantic | Durable facts and preferences that outlive runs | Document store, vector index | Search at time of need |
The distinction earns its keep in retrieval policy. Episodic memory answers “what did I already do,” and is queried by recency; semantic memory answers “what is true,” and is queried by relevance, typically through vector or hybrid search. Promotion between them should be deliberate: when a run’s outcome yields a durable fact, write it to the semantic store explicitly, rather than hoping a future transcript search rediscovers it.
Externalize the state
Context window (scarce) External state (durable)
─────────────────────── ────────────────────────
Goal + constraints ◄──── plan.md (intent)
Active subtask ◄──── progress notes (episodic)
Recent tool results ◄──── files, DB rows (artifacts)
Pointers, not payloads ────────► object store, DB (payloads)
│
▼
Resumable after any
restart or compactionThe strongest pattern for long-horizon work is to stop treating the transcript as the source of truth. The plan lives in a file the agent rewrites as it progresses. Intermediate products live in files or database rows, and the context holds pointers to them, not their contents. Progress is a ledger the agent updates after each meaningful step. Consider a workload that processes thousands of records over a horizon no context window survives: the only viable design keeps the processing state in the database, so any given model call sees only the slice of work in front of it. Externalized state also buys the property that matters most in production, which is resumability. A crash, a compaction, or a model swap costs you a working set, not a job.
Closing
Long-horizon agents fail on context before they fail on intelligence. Budget the window, compact to checkpoints that a fresh agent could resume from, separate episodic from semantic memory with distinct retrieval policies, and keep the truth in files and databases rather than the transcript. These are old systems disciplines pointed at a new runtime, which is precisely why they work.