Model routing in multi-agent workflows
July 2026 · 8 min read
Different agents in the same system should run on different models. A classifier doesn't need the same model as a drafting agent. Here's how to match each role to the right model — and why it matters for cost and quality.
Quick answer
Model routing means assigning each sub-agent in a multi-agent system its own model, matched to what that role actually demands — rather than using one model for everything. A triage agent deciding "is this urgent?" doesn't need the same model as a drafting agent writing a client-facing report. Getting this right cuts cost significantly and improves quality where it matters. Getting it wrong means either overpaying to run simple tasks on expensive models, or underpowering the judgment calls that needed real capability.
Why one model for everything is the wrong default
The instinct when building a multi-agent system is to pick one model — usually the best one available — and run every agent on it. It feels like the safe choice: if the frontier model is good enough for the hardest task, it's good enough for all of them.
This reasoning has two problems.
The first is cost. Frontier models (GPT-4o, Claude Sonnet, Gemini Pro) charge per token. A simple classification step — "is this email a support request or a sales inquiry?" — costs roughly the same whether you run it on a frontier model or a fast, cheap one like GPT-4o mini or Claude Haiku. At scale, those unnecessary tokens add up fast.
The second is speed. Frontier models are slower. Running every step of a pipeline through a model with higher latency slows the whole system down, including the steps where speed matters more than raw capability.
The other extreme — running everything on the cheapest model to cut costs — breaks the steps where capability genuinely matters. A drafting agent writing customer-facing copy on Claude Haiku will produce noticeably weaker output than one running on a stronger model. Cost optimisation at the expense of quality at the points that matter is just a different kind of wrong.
Model routing is the middle path: expensive models for the roles that need them, fast cheap models for the roles that don't.
What each role actually needs
The model requirement for a sub-agent follows from what its role asks it to do. Most roles fall into a few patterns.
Classifier and triage agents
These agents make categorical decisions: is this urgent, which queue does this belong to, what type of request is this? The answer is usually one of a small set of options, and the decision is typically low-stakes — it routes the item forward, it doesn't act on it directly.
These roles fit fast, cheap models well. GPT-4o mini, Claude Haiku, and Gemini Flash handle classification accurately at a fraction of the cost of frontier models, with lower latency. The key is a tightly written prompt with clear categories — don't ask a cheap model to infer what you mean.
Extraction agents
Extraction agents pull structured data from unstructured input: get the date, company name, and contract value from this email thread; extract the line items from this invoice. The task is well-defined and the output is structured.
Cheap models handle extraction well when the schema is explicit. The prompt does most of the work here — the model just needs to follow instructions reliably, which fast models do without the cost of frontier reasoning.
Research and summarisation agents
These agents read longer content — documents, threads, transcripts — and produce a summary or analysis. They need more capability than a classifier because the input is less structured and the output requires judgment about what's relevant.
A mid-tier model (Claude Sonnet, GPT-4o, Gemini Pro) is typically the right fit. Strong enough to handle nuance and longer context; not so expensive you're paying frontier rates for every summarisation step.
Drafting agents
Drafting agents produce text that a person will read — emails, reports, briefs, proposals. Quality is visible and matters. This is where underpowering has a real cost: weak output from a drafting agent is a weak output that goes to a customer or stakeholder.
Frontier or near-frontier models (GPT-4o, Claude Sonnet or higher) are appropriate here. The per-token cost is higher, but drafting steps are often less frequent than classification or extraction steps, and the quality difference is worth it.
Orchestrator and reasoning agents
These are the agents making judgment calls: should this escalate, which sub-agent should handle this next, is the output good enough to proceed? They need to reason across the full context of what's happened and make a decision that affects the rest of the pipeline.
This is where you spend money. Frontier models with strong reasoning — GPT-4o, Claude Sonnet, Gemini Pro — belong here. A wrong call at an orchestration point propagates downstream. A weak model making a poor routing decision can send the whole pipeline the wrong direction.
A worked example
Consider a lead qualification pipeline with five sub-agents:
| Agent | Role | Model choice | Why |
|---|---|---|---|
| Triage | Is this a real, qualified lead? | GPT-4o mini | Binary decision, runs on every lead |
| Research | What do we know about this company? | Claude Sonnet | Synthesis task, moderate context |
| Scoring | ICP fit score with reasoning | Claude Sonnet | Needs genuine reasoning, not just extraction |
| Router | Outbound / nurture / discard | GPT-4o mini | Low-stakes routing, clear criteria |
| Briefing | Write the rep brief | GPT-4o | Customer-adjacent output, quality matters |
Running all five agents on GPT-4o would cost roughly 3–4× more per lead than the routed version, with no quality improvement on the triage or routing steps — and likely worse speed on both.
The cost difference grows with volume. At 500 leads a day, routing correctly versus defaulting to one frontier model for everything is a meaningful budget line.
How to decide which model fits which role
A few questions get you to the right answer for most roles:
What is the output? A category, a structured object, or prose? Categories and structured output → fast model. Prose → stronger model.
How often does this step run? High-frequency steps (every item hits them) compound cost quickly. Low-frequency steps (only triggered on escalation) are less sensitive to per-token cost.
What's the cost of a wrong answer? A misclassification that gets corrected downstream by a gate is recoverable. A poor orchestration decision or a weak client-facing output is not. Higher consequence → stronger model.
How ambiguous is the input? Tightly structured, predictable input → cheap model handles it. Messy, varied, or high-context input → needs more capability.
Where Envelope fits
Envelope makes model assignment a first-class design decision. When you describe the multi-agent workflow you want, Envelope assigns models per sub-agent based on each role — so the routing happens at design time, as part of the spec, rather than being wired in code later or defaulting to one model for everything.
The result is a design you can review: every sub-agent has its role, its model, and the rationale for that assignment visible before anything is built. If the model choices need adjusting, you adjust the spec — not the implementation.
If you want to understand the full set of decisions that go into a well-designed multi-agent system — roles, pipelines, gates, and model routing — The anatomy of a multi-agent covers all of them.
Design your AI agents in Envelope
Envelope turns a plain-language description of your workflow into a complete AI agent system — agents with named roles, model assignments, tool access, handoffs, and human review gates. Free to start, no code required.
Frequently asked questions
What is model routing in multi-agent AI?
Model routing means assigning each sub-agent in a multi-agent system its own AI model, matched to what that role actually demands — rather than using one model for the whole team. A classifier gets a fast, cheap model; a drafting agent gets a capable one. The goal is matching cost and capability to the actual requirements of each role.
Do all agents in a multi-agent system need to use the same model?
No — and using the same model for every agent is almost always the wrong default. Simple roles like classification and extraction work well on fast, cheap models. Roles that involve drafting, judgment, or orchestration benefit from more capable models. Running everything on the frontier model overpays for simple tasks; running everything on the cheapest model underperforms on the hard ones.
Which AI model should I use for classifier or triage agents?
Fast, cheap models work well for classification: GPT-4o mini, Claude Haiku, Gemini Flash. The decision is typically categorical and well-defined, and latency matters because these agents run on every item in the pipeline. The key is a tight, explicit prompt — cheap models follow clear instructions reliably.
Which AI model should I use for drafting agents?
Frontier or near-frontier models: GPT-4o, Claude Sonnet or above, Gemini Pro. Drafting agents produce text that people read, so quality differences are visible and consequential. The higher per-token cost is usually worth it, especially since drafting steps tend to be less frequent than classification or routing steps.
How does model routing affect multi-agent system cost?
Significantly. A pipeline where every agent runs on a frontier model costs 3–5× more per run than one where cheap models handle classification and extraction and expensive models handle only drafting and orchestration. At scale, the difference becomes a real budget item. Model routing is the primary lever for controlling cost without sacrificing quality at the points that matter.
Can I change model assignments after a multi-agent system is built?
Yes, but it's easier to decide at design time. Changing a model assignment in a running system requires testing to confirm the new model handles that role's edge cases correctly. Getting it right in the design spec — before anything is built — is cheaper than diagnosing model-related quality problems in production.
How does Envelope handle model routing?
Envelope assigns models per sub-agent as part of the design spec. When you describe the workflow you want, Envelope makes model recommendations by role — so the routing decisions are explicit and reviewable before implementation begins, rather than defaulting to one model or being scattered across configuration files.