Where AI Coding Agents Waste Tokens and Why It Matters
Open any single request from a coding agent and it looks cheap. A few thousand input tokens, a few hundred output tokens, fractions of a cent.
Then the monthly invoice arrives and nobody can explain it.
The gap is not a billing error. It is the difference between what one request costs and what a session costs. Agents work in loops, and almost every property of a loop pushes token consumption up faster than the amount of work being done.
Short answer: Coding agents waste tokens in six places: context that gets resent every turn, reasoning tokens you pay for but never read, tool schemas and raw tool output that never leave the transcript, turns that repeat without making progress, output longer than the task needs, and one frontier model handling every turn regardless of difficulty.
Why agent costs grow faster than agent work
A chat request is one shot. An agent request is one turn in a conversation that the API has no memory of.
The Messages API is stateless. To continue a session, the harness resends the entire transcript on every turn: system prompt, tool definitions, every prior message, every tool result. Turn 20 carries everything from turns 1 through 19.
What you are looking at | What it measures | Why it misleads |
|---|---|---|
Cost of one request | Tokens in that single call | Ignores that the same prefix is sent again next turn |
Average cost per request | Total spend divided by call count | A long session and a short one average into the same number |
Tokens per session | The real unit of agent cost | Rarely tracked, because billing is per request |
That resend is the engine underneath most of the waste below. Prompt caching softens it, and we get to that. It changes the price of the resend. It does not change the fact that the resend happens.
1. Context you resend on every turn
Because history accumulates and gets resent, the volume of tokens submitted grows with roughly the square of the turn count, while the actual new work per turn stays flat. Caching changes what most of those tokens cost. It does not change how many of them get sent.
An illustrative session makes the shape visible:
Illustrative session, 2,000 new tokens per turn, no fixed prefix | Turn 5 | Turn 10 | Turn 20 |
|---|---|---|---|
Transcript size sent that turn | 10,000 | 20,000 | 40,000 |
Cumulative tokens submitted | 30,000 | 110,000 | 420,000 |
New tokens actually added | 10,000 | 20,000 | 40,000 |
Prompt caching is the main defense. On Anthropic's published rates, cached reads cost roughly 0.1x the base input price and cache writes cost 1.25x at the default five minute TTL, so a stable prefix pays for itself by the second request. Other providers price caching differently, but every implementation shares the same constraint. The catch is that caching is a prefix match: one changed byte anywhere in the prefix invalidates everything after it.
Watch for a timestamp, a session ID, or a per-user string interpolated into the system prompt. That single line makes every downstream token uncacheable. Decides whether your prompt assembly code needs reordering before anything else is worth tuning.
2. Reasoning tokens you pay for and never read
Extended reasoning is billed as output. On current models it is on by default, and whether the reasoning is displayed, summarized, or omitted changes nothing about the price. Teams that hid reasoning from their UI often assume they stopped paying for it. They did not.
This is not an Anthropic quirk. Some models make it structural: in our three-model workflow analysis, Kimi K2.7 turned out to have mandatory thinking with no way to disable it, so simple tasks still pay for reasoning they cannot use.
Reasoning also competes with the answer for the same output budget, which produces a specific and expensive failure: a response that is mostly thinking, truncated mid-answer, and then retried from scratch.
Setting | Effect on tokens | Effect on what you see |
|---|---|---|
Reasoning omitted | None. Billed in full. | Empty reasoning blocks, looks like a pause |
Reasoning summarized | None. Billed in full. | A readable summary |
Lower effort | Fewer reasoning tokens, fewer tool calls | Terser, more direct work |
Watch for responses ending on a token limit rather than a natural stop. Decides whether to raise the output ceiling or lower the effort level, which are opposite fixes for the same symptom.
3. Tool definitions and tool results
Tool schemas render at the very front of the request, before the system prompt. A catalog of forty tools sits in front of every single turn whether the agent uses one of them or none.
Worse, adding or removing a tool mid-session invalidates the entire cache, because the change lands at position zero of the prefix.
Tool results are the other half. A grep across a large repository, a full file read, a verbose stack trace: all of it enters the transcript verbatim and stays there for the rest of the session, resent on every subsequent turn.
Pattern | What it costs | Better shape |
|---|---|---|
Forty always-loaded tool schemas | Fixed tax on every turn | Tool search with deferred loading |
Reading a whole file to change one line | Full file in context, permanently | Targeted read plus a scoped edit |
Raw command output dumped into context | Grows the resent prefix forever | Filter before the result lands in context |
Adding a tool mid-session | Full cache invalidation | Declare tools up front |
Watch for tool results larger than the code change they informed. Decides whether the fix is a narrower tool surface or a filtering step between the tool and the transcript.
4. Turns that repeat themselves
This is the most expensive category, because it produces nothing.
An agent that is stuck does not look stuck. It keeps working: rerunning the same test, rereading the same file, trying a variation that fails the same way. Every one of those turns carries the full accumulated transcript, and every one adds to it.
Signal in the trace | Normal | Evidence of a stall |
|---|---|---|
A failed command | Expected, agents recover | The same failure returning repeatedly |
Rereading a file | Fine once | Third read of an unchanged file |
Test output | Changes as the fix lands | Byte-identical across attempts |
Edits made | At least one per few turns | Several turns with no edit at all |
The distinction that matters is not failure versus success. It is whether the failure is changing. One failed command is ordinary. The same failure returning while the agent keeps moving is a loop, and loops are where sessions quietly triple in cost.
Watch for sessions in the long tail of your token distribution. Decides whether a task needs a stronger model, a better prompt, or a hard stop.
5. Output that says more than it needs to
Output tokens are priced several times higher than input tokens. Claude Opus 4.8 runs $5 per million in and $25 per million out, a 5x gap, and most frontier models are shaped similarly. Verbosity is expensive per token even when the volume looks small.
Three habits dominate:
Whole file rewrites where a scoped edit would do, which pay for the entire file in output tokens.
Narration between tool calls, restating what was just done and what is about to happen.
Long written deliverables, where a summary grows sections nobody reads.
Every one of those tokens is also an input token on the next turn, because the response joins the transcript. Verbose output is billed twice: once at the output rate, then repeatedly at the input rate for the rest of the session.
Watch for assistant messages that are longer than the diffs they produced. Decides whether the prompt needs an explicit conciseness instruction, which is usually a cheaper fix than changing models.
6. One model for every turn
The five sources above are about volume. This one is about price per token, and it is the largest single lever.
A coding session is not one kind of work. It is many:
Interpret an instruction
Search the repository
Read code
Form a hypothesis
Edit a file
Run tests
React to the result
Some of those turns genuinely need the strongest available model. Most do not. Reading a file and reporting what is in it does not require frontier reasoning, and paying frontier prices for it is a pure transfer from your budget to your provider's revenue.
Turn type | Capability actually needed | Typical model choice |
|---|---|---|
Mechanical continuation after a file read | Low | Frontier, by default |
Repository search and summarization | Low to moderate | Frontier, by default |
Root causing a race condition | High | Frontier, correctly |
Planning a change across several files | High | Frontier, correctly |
Rerunning a test and reporting the result | Low | Frontier, by default |
Picking one model at the start of a session and keeping it for every turn is a static answer to a workload that changes turn by turn.
Route the turn, not the session
This is the problem Entelligence Model Router was built for. Instead of choosing a model once, it makes a fresh choice per turn based on the work in front of the agent: the least expensive model that can handle the current turn, escalating when the trajectory shows a stall, and stepping back down when progress returns.
The three lanes matter more than the two most teams assume. Work is not simply easy or hard. Our own three-model analysis found a distinct middle tier that most routing schemes miss:
Lane | The work it fits | What happens without it |
|---|---|---|
Efficient | Bulk mechanical turns: renames, file reads, test reruns | Frontier pricing on work a cheap model finishes identically |
Middle | Long-horizon, repo-scale work needing real code comprehension | Teams overspend by defaulting up, or underserve by dropping down |
Frontier | Ambiguous requirements, architecture, security-critical reasoning | Quality gaps that surface in production |
The prompt cache constraint from section 1 shows up here too, and it is the part naive routing gets wrong. Caches are scoped to a model, so switching models mid-session means re-ingesting the transcript from scratch. Deep into a session, staying on a warm model for one more turn can genuinely cost less than moving to a nominally cheaper one. The router compares those paths before switching.
On 89 Terminal-Bench 2.1 tasks under an identical Claude Code harness, we measured:
Configuration | Tasks solved | Total cost | Cost per solved task |
|---|---|---|---|
Entelligence Router | 71 / 89 | $65.75 | $0.93 |
Claude Opus 5 | 63 / 89 | $190.62 | $3.03 |
Claude Opus 4.8 | 58 / 89 | $155.33 | $2.68 |
That is a vendor benchmark on a single seed, with run to run variance of roughly two tasks, and it is not a universal model ranking. The full methodology and its four limitations are in the benchmark write-up. What it does show is that turn level model placement beat one static choice on both cost and completion, which is the opposite of the tradeoff most teams assume they are making.
What to change first
Not all of these cost the same to implement:
Change | Effort | Where it helps |
|---|---|---|
Freeze the cacheable prefix, move volatile strings later | Low | Section 1 |
Add a conciseness instruction to the system prompt | Low | Section 5 |
Tune effort level per route instead of using one default | Low | Section 2 |
Filter tool output before it enters the transcript | Medium | Section 3 |
Alert on sessions in the top decile of token spend | Medium | Section 4 |
Route model choice per turn | Medium | Section 6 |
Start with measurement, and with the five metrics worth tracking. Entelligence research across more than 1 million pull requests and 2,444 organizations estimates that about $0.18 of each dollar spent on AI coding tools becomes shipped product. That is vendor produced research rather than a universal benchmark, but it points at the same thing this article does: the spend is real, and most teams cannot yet see where it goes. Agent Insights is where we put that visibility, broken out by outcome, project, and model, including the untracked spend sitting in projects with no budget attached.
Frequently asked questions
Why is my AI agent so expensive when each request looks cheap?
Because the API is stateless. Every turn resends the full transcript, so the volume of tokens submitted grows with roughly the square of the turn count while the new work per turn stays flat. Caching reduces what most of those tokens cost, but the growth in volume is what makes cost per request the wrong unit. Cost per session is the right one.
Does prompt caching solve token waste?
It reduces the price of resent context substantially, with cached reads at roughly 0.1x the base input rate. It does not reduce the volume, and it breaks entirely if anything in the prefix changes. Caching is necessary and not sufficient.
Are reasoning tokens billed even when they are hidden?
Yes. Display settings control visibility, not billing. Hiding reasoning from your interface changes nothing on the invoice. Lowering the effort level is what reduces reasoning spend.
What is the single biggest source of waste?
Turns that repeat without progress, because they pay the full accumulated context cost while producing nothing of their own. Model over-selection is the larger lever on price per token, but a stalled loop is the only category where the spend has no deliverable attached to it.
Do I need a router, or can prompting fix this?
Prompting fixes verbosity, over-verification, and unnecessary tool calls, and it is cheaper to try first. It cannot change which model serves a turn. If your traces show routine work running on frontier capacity, that is a routing problem, not a prompting one. Our comparison of nine LLM routers covers the options, including the self-hosted ones.
How do I tell a stalled agent from a hard problem?
Look at whether the failure is changing. Different errors across attempts mean the agent is exploring. The same error while the agent keeps acting means it is looping. That distinction is visible in the trajectory long before it shows up on the invoice.
Measure the session, not the request
The reason agent costs surprise people is that every instinct trained on API pricing is calibrated to the wrong unit. Per request, everything looks fine. Per session, the same workload can cost three times what it should.
Six places to look, in order of how much they usually cost: repeated turns, model over-selection, resent context, tool output, reasoning, and verbose responses. Most teams can measure none of them today.
If you want the cost side handled at the gateway rather than in your prompts, see Entelligence Model Router in action.