Entelligence Model Router

This started with us reading the traces, the turn-by-turn logs, behind the pull requests our agents write, trying to work out why the bill looked the way it did. Mostly what we found was Opus, the priciest model in the lineup, doing grunt work. A very expensive shell script.

The turns where Opus was actually being Opus were rare, maybe a handful per session: the root cause, the fix that can't break three other things. The rest was plumbing. Open a file, search the codebase, rerun the tests, trim the conversation to fit, write the PR description. All of it billed to whatever model the agent asked for when the session started.

Model Router is the layer we ended up putting in the middle. Whatever harness you run still asks for Opus 4.8, GPT-5.5, or whichever frontier model it wants; the agent API doesn't change, and nothing looks different from the agent's side. The router reads the turn that's actually in front of it and decides which model should serve it.

It ran on our own work first. About 60% of our internal PRs now come from agents going through this path, and that's still the evidence we trust most, not because the number is big but because the workload is messy in the right way: old branches, half-broken test suites, long cached sessions, agents wandering around the repo looking for things. If the router were quietly making that work worse, we'd have switched it off weeks ago.

Benchmarks are the cleaner version of the same check, so we ran one. The router ships with two presets, Balanced and Eco. Balanced is the one that's supposed to match Opus, and on a 45-task Terminal-Bench slice it did: same pass count as direct Opus 4.8, roughly half the money. Eco gives up some quality for a much lower bill and landed under Opus but above our Sonnet baseline (Anthropic's mid-tier model), at about a quarter of the Opus cost. openrouter/auto solved fewer tasks than any of them.

To be clear about what the chart shows: these are the provider bills from the run, not a per-token price sheet. The distinction matters more than it looks, because a cheap model that loops isn't cheap, and prompt caching bends the whole bill once sessions get long.

Balanced solved 25 of 45 for about $16.50. Direct Opus 4.8 also solved 25, for $32.67. Eco got 22 at $8.43. Sonnet is the odd row: around 17 solves for $29.96, nearly Opus money. The mechanics are mundane. A weaker model takes more turns per task, every turn re-reads a longer transcript, and the tasks it never cracks burn their full budget anyway. Its per-token price is a fraction of Opus's; its cost per solve came out higher. openrouter/auto solved 16 at $11.67.

What we ran

Terminal-Bench felt right for this because it's close to how the agents actually get used. The agent gets a prompt and a terminal, and at the end a hidden grader either passes the work or it doesn't.

Our runs shared the same harness and prompts, prompt caching on, roughly a 15-minute cap per task. The slice is deliberately mixed. Easy repairs and parser jobs on one end; async bugs, crypto, bioinformatics on the other. The hard end is there to keep the router honest, since even Opus misses plenty of those.

Router against router

We also put OpenRouter's openrouter/auto on the same 45 tasks, since it's the obvious thing to compare a router against. It picks a model per request from a pool of frontier models. It solved 16.

The shape of that run is worth a look: 210 calls, 89% of them to GPT-5.5, the rest to smaller Gemini models. And most of its misses weren't wrong answers. 22 of the 29 unsolved tasks hit the agent budget before finishing anything.

The cost line needs a footnote too. OpenRouter billed $11.67, about a third of the Opus run, but that's their billed price where our numbers are raw provider spend, and our runs were capped at 15 minutes a task while theirs ran uncapped. Read it as a rough position, not a precise one.

Inside the Router

The agent doesn't know the router exists. That was a rule from day one, and it made the implementation harder while keeping the product simple: no special prompt, no alternate tool API, nothing for anyone to integrate. The proxy reads the transcript and treats it as telemetry.

As the trace builds, the router watches what the agent is doing and whether it's getting anywhere. Most turns stay on cheaper capacity. Frontier, the Opus class of models, is saved for the turns that have earned it.

The hard part is restraint. A lot of turns are setup or routine execution, a few genuinely need frontier reasoning, and the router has to tell them apart from the trace itself rather than from a keyword match.

The switch was messier than we expected, honestly. The incoming model inherits a transcript full of tool calls it never made, every provider has its own opinions about tool pairing and reasoning blocks and sampling settings, and you land on a cold prompt cache, which means the first turn after an escalation gets billed at full input price. The router does that bookkeeping. The cold turn is also one more reason not to escalate casually.

Doesn't switching wreck the cache?

Once people saw per-turn routing, the same three questions kept coming back. Doesn't this kill prompt caching? Does context get lost when the model changes? How does the next model know what happened without re-reading everything at full price?

The knot unties once you separate deciding from switching. The router makes a decision every turn, but the decision is usually "stay." Sessions are sticky by design, and a switch is a rare event with a reason attached. A run that never gets stuck can go start to finish on one model, warm cache the whole way.

And context was never in the cache to begin with. These APIs are stateless: the agent sends the full transcript with every request, and the cache is just the provider not re-billing the part of the conversation it has already seen. Whichever model we route to gets the same transcript the previous one would have gotten. Nothing is lost; the only thing that changes is the price of the first read.

So the cache math is per model, not per session. Staying is cheap warm reads. Switching is one cold read on the new model, then warm again, and the old model's cache doesn't vanish the moment we leave it. That cost is priced into the escalation decision, and compression keeps the re-read small: the transcript that lands on the new model is the tidied one, not every raw tool dump in full.

Examples from the run

The aggregate score is one thing, but we learned more from reading individual traces. A few are below, including one we're not proud of.

Trace routing replay

There is an up-front predictor and it's useful, but it's a guess, and we don't want a guess committing the most expensive model for a whole session before the run has shown anything.

The better signal shows up later, in the trace itself: the same failure twice, test output that isn't changing, progress that's flattened out. Repetition alone isn't it, though. Rerunning tests while they go from red to green is normal work; you're only stuck when the results stop changing while the actions keep coming. And a stuck verdict has to expire, because one rough patch shouldn't pin a two-hour session to frontier pricing. Some misses still slip through. The annoying kind is when the agent confidently declares the job done and the tests disagree.

Prepared trace route, all steps exposed

Results

Cost per solved task is the cleanest way we've found to put it. Balanced came out around $0.66 a solve, Eco $0.38, openrouter/auto $0.73, Opus 4.8 $1.31, and Sonnet $1.76.

We still like frontier models. We just stopped treating every turn as frontier-worthy by default.

The thing we keep coming back to is that the model picker is the wrong place to spend attention. Nobody wants to choose a model per prompt, and a static default is wrong in one direction or the other, either overpaying for routine turns or starving the hard ones. What a turn needs is already sitting in the transcript. Something just has to read it.

Availability

Entelligence Model Router opens to enterprise customers. If your coding-agent bill has started to look like an infrastructure line item, ask us for access.