Every team building an AI code review product right now is turning the same dial. Buy a bigger model, raise the reasoning budget, wait for the next frontier release. We tried all three. Then we made AI code review 3.4× faster, 65% more thorough and 50% cheaper, and the model swap turned out to be the last thing that moved the number, not the first.

This is the teardown of how we made AI code review 3.4× faster and 50% cheaper across thousands of production pull requests. Wall clock from webhook to posted comment fell from 7.4 minutes to 2.2. Hand-verified precision went from roughly 73% to 94%. Findings per PR climbed from 0.88 to 1.45. Six levers, ranked by what each actually returned: agentic workflow, evaluation loop, prompt design, reasoning budget, agent turns, model choice. The one most teams reach for first came last. Full measurement notes are in the original engineering write-up.

Why review latency is now the expensive part

Generation got cheap. Verification did not. Google Cloud's 2025 DORA report found AI adoption has reached 90% of surveyed software professionals and, for the first time, correlates positively with delivery throughput. It also found that AI adoption still correlates negatively with delivery stability. More change, shipped faster, into systems whose control loops did not change.

The queue moved to review. Telemetry from Faros AI across roughly 22,000 developers put median time in PR review up 441% in its 2026 dataset, against 91% a year earlier. And METR's randomized controlled trial found experienced open-source developers took 19% longer on real tasks with AI tools while believing they had been 20% faster, which is a calibration problem before it is a tooling problem.

So automated code review bot that burns seven minutes per pull request and posts noisy comments is not neutral ovn aerhead. It is the bottleneck, wearing a helpful hat. That framing decided which levers we pulled.

More reasoning bought us more spend

We ran four production experiments against real customer pull requests. Three cost more and changed nothing measurable.

Figure 1. Four reasoning and ensemble experiments, indexed to baseline cost. Only the cheapest configuration shipped.

• Maximum reasoning mode. 572 reasoning tokens per call against a 24 to 46 baseline. 2.8× the cost, no measured gain on per-file defect detection.

• Three-pass ensemble. Correctness, security and robustness as separate agents, each re-reading every file. 3.0× the cost, matched by a single pass.

• Fifty percent more agent turns. 1.37× the cost, reverted. Agents grind to any ceiling you give them.

• Multi-angle single pass. 1.0×. Shipped.

What did move the number was subtraction. We cut the system prompt from 38,500 characters to 3,400, deleting every worked example and trap catalogue, and hand-verified precision went from about 73% to about 94%. If you are fighting AI code review false positives, the instinct is to add more instructions describing the traps. In our benchmark that instinct was the cause, not the cure. Fewer tokens in, better output out.

Two stages decide everything

Six of the eight stages in our review pipeline call a model. Only two bound the outcome, and neither one improves when you buy a bigger model.

Figure 2. The review pipeline. Triage sets the recall ceiling, reflection sets the precision floor. Everything between them is throughput.

Triage is the recall ceiling. It decides which changed files a reviewer ever sees, and a file dropped there cannot be recovered by any model downstream. Reflection is the precision floor. It attacks every finding against the real code, and most findings die there. That adversarial pass, not the model, is where 73% to 94% precision came from.

The failure this explains. For months our triage and dedup ran on a cheap helper model while the reviewing agents ran on the frontier one. Triage silently capped recall. Dedup collapsed and posted five comments describing one root cause on a customer PR. Putting the whole pipeline on one model took that same PR to six distinct findings, at no extra cost per token.

This is the part buyers of any AI code review tool should interrogate hardest. Cheap helper models on the gating stages are invisible on a cost dashboard and catastrophic in the PR conversation.

Read the file once

The investigation is expensive. The perspective is nearly free. Most teams pay for both. A three-pass ensemble reads the file, traces the call chain, then throws that context away and does it twice more under a different heading. A multi-angle single pass reads once and applies correctness, security and robustness lenses to context it already holds. Ensemble-grade coverage at a third of the price, and the largest cost reduction in the reviewer that had nothing to do with changing models.

The same principle runs one level down. Before the agent starts, we resolve the callers of every symbol the diff removed with ripgrep and hand it the answers. Concrete caller data instead of hoping it greps. Work a lookup can do deterministically should never cost an agent turn, and every turn it saves comes straight off the critical path. That is what context aware code review should mean in practice: precomputed structure, not a bigger context window and a hopeful prompt.

Context is billed in dollars, output is billed in seconds

When we did finally change the model underneath, the amount of context we sent barely moved. What collapsed was generated tokens and round trips.

Figure 3. Per-review token, call and latency deltas, plus the cost split by line item. Prefill is parallel, generation is serial.

Input tokens fell 9%. Output tokens fell 54%, from 58,200 to 26,800. Serial model calls fell 40%, from 127 to 76. Wall clock fell 70%. We sent essentially the same context and got a better review back in under a third of the time, which is not a pricing story. Prefill is parallel and generation is serial, so cutting output tokens and round trips is the latency result. The property that paid was efficiency, not power. The same benchmark that rewarded a terser model had already refused to reward a stronger one.

The cost split surprised us. 59% of every dollar went to cache writes, 28% to cache reads, 13% to output, and 0.2% to fresh uncached input. Under Anthropic's prompt caching model, a cache write costs 1.25× the base input rate and a cache read costs 0.1×, so with 86% of our input served from cache, re-reading a large repository is nearly free. Establishing that cache is the single biggest line item on the bill. The lever most teams reach for is trimming context, and that is not where this bill lives.

Cache writes are a turn count, not a prefix

Fifty-nine percent of spend on cache writes reads like a prompt-stability problem. Regressed against 1,384 reviews it is something else entirely: cache_write ≈ 7,690 × calls + 76,306, with r = 0.957. About 7.7K of cache write per model call, flat at every prompt size. 89% scales with turns and only 11% is prefix. Each turn appends a tool result, and that increment is cache-written exactly once.

That moves the target. The 59% block is paid per turn, which puts 93% of the bill in reach of anything that stops the agent searching. So we stopped making it search. Every repository is parsed into a structural graph of functions, classes, imports, call edges and test coverage, and one query returns the blast radius of a diff in 636 tokens and 5 milliseconds. Naming the edge, not just ranking the file, is what makes it cheap, because it lets the agent skip a file instead of opening it to discover it was irrelevant.

• Reviewer tokens per review: 4.54M down to 3.20M, a 30% cut.

• Cost per review: $0.284 down to $0.208, a 27% cut.

• Search turns per agent: 8.9 down to 6.2, the thing actually being removed.

• Break-even is 0.72 turns saved per review, against a 29× margin. The graph artifact is 13.5MB per repository and 0.5% of the saving keeps it current.

One guardrail matters more than the saving. Comment volume and acceptance rate have to hold. A graph that cuts tokens by making the agent incurious is a regression dressed as a saving, and it would pass a cost dashboard cleanly. Acceptance, not spend, is the number this ships against.

Nothing ships on one good run

Run-to-run variance on an agentic reviewer is large enough that almost any change looks like a win if you measure it once. A candidate clears four gates or it does not reach the fleet. A benchmark across 10 repositories, with ground truth drawn from two rival commercial review bots and scored per repository rather than on average. A human hand check of every comment against its diff, after we dropped LLM judges as the least reliable part of the whole evaluation. A one-org shadow rollout where candidate and incumbent run in the same production fleet in the same hour, so there is no deploy to revert. Then a daily per-review, per-model signal that becomes the next candidate's baseline.

The clearest argument for that harness is the change we did not ship. A nudge pushing agents to re-sweep for a second defect looked like a clean +30% recall win. One run, high variance, +49% latency. It is still off. A loop that ships every apparent win is a slower way of guessing. Our published AI code review benchmark is built on the same gates.

Four questions to ask any AI code review platform

• Which stages run on which model? If triage or dedup runs on a cheap helper model, recall is capped before any reviewing agent sees the diff.

• Is precision hand-verified or LLM-judged? We found LLM judges the least reliable component in our evaluation. Ask for the hand-checked number.

• Where does the cost actually sit? If a vendor's answer to AI code review cost is trimming context, they have not measured cache write against turn count.

• What happens after a bug ships? A reviewer that reads only the diff cannot know that this class of change took production down last quarter.

Closing the loop

Every number above came from the harness, not the model. That is the durable asset: a benchmark, a rollout gate and a daily signal, with providers selectable behind one switch. A model migration is a step change you get once. The harness compounds every week you keep measuring it.

It is also only half of the loop, and the half most tools stop at. Entelligence started as a diff reader too: parse the new code, flag the bugs, move to the next PR. What we learned is that bugs caught are not the same thing as incidents prevented. So Entelligence now reads the whole codebase and the incident history alongside it. Historical incidents feed an operational memory that maps breakage patterns to the code that caused them, and every new pull request is matched against that record. The reviewer does not just say this looks wrong. It says this is what broke last time.

That is what production-aware code review means, and it is why the loop compounds instead of plateauing. As PR volume rises, the AI code review agent gets better rather than noisier, because every incident becomes searchable precedent. Observability, review and remediation stop running as three disconnected agents in three separate tools, and the codebase itself starts learning as the engineering team scales. The point is not a faster comment. The point is that the same class of bug never ships twice. The 3.4× and the 50% are what make that loop cheap enough to run on every pull request instead of the risky ones. See how the economics work across a whole fleet.