The Token Economics of Agentic Systems
Pull-quote: “A chatbot pays for a conversation. An agent pays for a loop, and the loop re-reads its entire history on every turn. Per-token pricing is linear; agent cost is quadratic in ways the invoice only reveals at the end of the month.”
Why this matters
Teams price agent features by looking at per-token rates and estimating a prompt or two. Then the bill arrives at 10–50x the estimate, and the postmortem always finds the same culprits: not expensive tokens, but token volume nobody modeled. Agent cost is an architectural property, and it is controllable, but only if you measure where it actually accrues.
Where the money goes
One agent task, anatomized
──────────────────────────
turn 1 [system + tools + task........] input
turn 2 [system + tools + task + t1...] input (re-read)
turn 3 [system + tools + task + t1 + t2...] input (re-read)
turn 4 [........ + 40KB tool output pasted ....] input (spike)
turn 5 [retry after tool failure — full re-read] input (waste)
⋮
turn N context near limit; model summarizes itself input + output
│
▼
Input tokens ≈ 90%+ of spend. The task paid for its own history N times.Three structural drivers dominate: loops (every turn re-submits the growing history), retries (a failed tool call replays the entire context for another attempt), and context stuffing (raw tool outputs, entire files, full retrieval results pasted where a summary would do). Output tokens, the thing most people watch, are usually a rounding error next to input volume.
The levers, in order of return
| Lever | Mechanism | Where it pays |
|---|---|---|
| Prompt caching | Stable prefix (system prompt, tool schemas) cached by the provider at a discounted rate | Every multi-turn loop; often the single largest saving |
| Compaction | Summarize or truncate history past a threshold; keep decisions, drop transcripts | Long-running tasks that would otherwise grow quadratically |
| Tool-output hygiene | Return structured extracts, not raw dumps; paginate; cap sizes at the tool boundary | Any agent that reads files, logs, or web pages |
| Model routing | Small model for classification, extraction, and routine steps; frontier model only where judgment is required | Pipelines with mixed-difficulty steps |
| Batch vs interactive | Queue non-urgent work to batch endpoints at reduced rates | Evals, backfills, enrichment, nightly jobs |
| Loop budgets | Hard caps on steps and spend per task; stop conditions that detect no-progress loops | The pathological tail that dominates the invoice |
Order matters. Caching and compaction attack the quadratic term; routing and batching attack the unit price; budgets cap the tail. Most teams reach for routing first because it feels like engineering, but caching a stable prefix is usually worth more and costs an afternoon.
The only metric that survives contact with finance
Cost per token is a procurement number. The number that matters is cost per completed task: total spend on an outcome, including every retry, every abandoned loop, and every run that failed. Two consequences follow immediately. First, a cheaper model that fails more often is frequently the expensive option once its failures and retries are billed to the task. Second, reliability work is cost work: cutting a 20% failure rate does more for unit economics than shaving prompt tokens. You can only see this if your tracing attributes cost per span and rolls it up per task, which is why the economics work depends on the observability work.
When the unit of value is a deliverable rather than an API call, meter it that way: an append-only ledger that ties generation spend to each produced artifact gives the question “what did this deliverable cost?” an auditable answer rather than a monthly guess.
Closing
Agent economics reward the teams that treat cost as an engineering property, not a billing surprise. Measure where tokens actually accrue: loops, retries, re-reads. Cache the stable prefix, compact the history, keep tool outputs disciplined, route by difficulty, batch what can wait, and cap the pathological tail. Then judge the whole system on cost per completed task, because that is the number your CFO is going to compute whether you do or not.