How to Route Models Inside a Long-Running Coding Agent

Most teams start with the simplest agent design: pick a strong model and let it do everything.

That works for demos. It gets expensive and messy once the agent runs for longer tasks.

A coding agent is not one workload. It is a loop:

1. understand the request

2. inspect the repo

3. choose files to edit

4. modify code

5. run commands

6. read failures

7. retry

8. report evidence

9. wait for acceptance

Each step has a different cost and reliability profile. That is why GPT-5.6-style model tiers are interesting. The practical lesson is not simply "use the newest model." The lesson is to route agent work by role.

A simple role split

The important part is separating implementation from acceptance. The model that wrote the patch should not be the only authority saying the patch is done.

What to measure

Do not optimize only for price per token. For agents, the better metric is cost per accepted task.

Track:

• input tokens per task

• output tokens per task

• tool calls per task

• retry rate after failed commands

• p95 time to passing validation

• cache hit rate on stable repo context

• cost by role: planner, worker, verifier, utility

• reports that claim success without command evidence

This makes routing decisions more concrete. If log parsing burns too much expensive context, move it to a utility model. If workers loop too often, upgrade the worker or tighten the planner constraints. If verification misses failures, separate the verifier from the worker.

Cache the stable context

Repo-heavy agents repeat a lot of stable context: architecture notes, coding conventions, test commands, package layout, and API contracts.

Keep that context cache-friendly when the API supports it. Then keep volatile context short:

• current error

• changed files

• latest command output

• remaining acceptance criteria

A lot of teams think they have a model pricing problem when they really have a context layout problem.

Guardrails before autonomy

If an agent can run for a long time, it needs boundaries.

Start with:

• disposable workspace

• no production secrets

• command allowlist

• network access off unless needed

• per-task spend cap

• validation commands defined before implementation

• final report with actual command output

• separate approval before merge

This setup is boring on purpose. It lets the agent keep working without quietly increasing the blast radius.

Routing policy example

A minimal policy could look like this:

The goal is not to make the workflow complicated. The goal is to make it observable.

If you can measure each role, you can improve each role. If one model does everything, you only know the agent was expensive or wrong after the fact.

Final thought

The next agent upgrade is probably not replacing every call with the newest frontier model. It is building a workflow where the newest model can be inserted only where it changes the outcome.

That means routing, caching, validation evidence, and hard limits.

Question: what do you currently log for your coding agents: token cost, tool calls, retries, validation output, or something else?