The anatomy of a multi-agent system

Quick answer

A multi-agent is a small team of specialised AI agents that work together on a task too broad for one agent to handle well. Each agent has a narrow role, its own model and tools, and a defined handoff to the agents around it. The team runs as a pipeline — sometimes ordered, sometimes concurrent — with human checkpoints ("gates") built in wherever a decision needs sign-off before it goes further. Done well, it looks less like "an AI" and more like a small, accountable department.

That's the short version. The rest of this piece is the anatomy — the actual parts, what each one does, and why skipping the design step is the reason most multi-agent attempts fall apart.

Why one agent isn't enough

The instinct when you first automate something with AI is to reach for one agent and one long prompt. Give it the context, give it the tools, ask it to handle the whole job. This works for narrow, single-step tasks. It stops working the moment the job has more than one kind of decision in it.

A single agent asked to "handle customer support" is quietly being asked to triage, research, draft, decide tone, escalate, and follow up — all inside one context window, with one set of instructions trying to cover every case. The result is usually mediocre at all of it: cautious where it should be fast, verbose where it should be terse, missing escalations it was never really built to catch.

Specialisation fixes this the same way it fixes it in a human team. A triage agent that does one thing well beats a generalist agent doing six things adequately. The case for a multi-agent isn't "more AI." It's narrower jobs, done properly, handed off cleanly.

The sub-agent: the unit of design

If the multi-agent is the team, the sub-agent is the person. Each sub-agent is a single, narrowly defined role with five things that fully describe it:

• Prompt — what it's told to do, in plain instructions. Not the whole job; its slice of it.
• Role — what it's responsible for and, just as importantly, what it isn't. A "Researcher" isn't also the "Writer."
• Capabilities — the specific things it's allowed to do. Search the web, read a CRM record, draft an email. Capabilities are how you keep a narrow role narrow in practice, not just on paper.
• Model — which model actually runs this sub-agent. More on this below, but the short version: not every role needs the same model.
• Access policy and reporting line — what data and systems it can touch, and who reviews or receives its output (reportsToKey in Envelope's schema — the sub-agent above it in the chain).

This is the part most people skip. They wire up a few prompts, call it a team, and wonder why nobody can tell which agent is responsible for what six weeks later. A sub-agent with a defined role, a defined capability set, and a defined report line is auditable. A prompt with vague instructions bolted to a shared toolset is not.

Model routing

Not every sub-agent needs the same model, and treating them as if they do wastes money and, more importantly, misses the point of building a team in the first place.

A classification or extraction sub-agent — deciding "is this urgent," pulling a field out of a document — usually doesn't need your most expensive model. A drafting sub-agent writing customer-facing copy might. A reasoning-heavy sub-agent making an escalation judgment call probably needs the strongest model you can afford for that one job.

This is model routing: matching each sub-agent's model to what its role actually demands, rather than picking one model for the whole team. It's a cost decision and a capability decision at the same time. Route badly and you either overpay running your cheapest tasks on your best model, or underpower the one decision in the pipeline that actually needed judgment.

The pipeline: ordered vs. concurrent

A pipeline is how sub-agents are wired together — the order they run in and what depends on what. There are two basic patterns, and most real pipelines use both.

Ordered steps run one after another because each one needs the previous one's output. A Researcher has to finish before a Drafter can write anything grounded in what it found. This is expressed as a dependency (dependsOn in Envelope's schema) — step B doesn't start until step A is done.

Concurrent steps run in parallel because they don't depend on each other. If you're scoring a lead on fit and separately checking whether it's a duplicate, there's no reason those two sub-agents should wait on one another. Running them concurrently isn't just faster — it's a correctness signal. If two steps genuinely don't need each other's output, they shouldn't be artificially chained.

Getting this wrong in either direction causes real problems. Force everything into a strict sequence and you get a slow, brittle pipeline where an unrelated bottleneck holds up the whole team. Run dependent steps concurrently by mistake and a sub-agent acts on information that hasn't arrived yet. The pipeline shape is a design decision, not a default.

Human-in-the-loop and gates

This is the part that's easiest to treat as an afterthought and the part that matters most once a multi-agent is doing anything that carries real consequence.

A gate is a checkpoint where the pipeline stops and waits for a human decision before continuing. Not a notification. Not a log entry someone might check later. An actual pause — the agent that would send the email, update the record, or trigger the next step waits until a person approves, rejects, or edits what's in front of them.

Deciding where to put a gate is a design decision, the same way deciding a sub-agent's role is. Ask: what's the cost of this step being wrong, and can it be undone? Sending an internal draft for review costs little either way. Sending an external email to a client, changing a billing record, or triggering an irreversible action is exactly where a gate belongs — before the action, not as a review of it afterward.

A properly designed gate also needs to define what happens if nobody responds. A gate with no timeout behaviour just becomes a queue of stalled work. A gate with a defined timeout and a clear on-reject path — does it stop, retry, or fall back to a different sub-agent — keeps the pipeline moving without removing the human's authority to say no.

This is one of the clearer differences between a real multi-agent and a chain of prompts. A chain of prompts either runs unattended or gets manually checked after the fact. A properly designed team has the checkpoints built into the structure, at the points where they were deliberately placed.

The stitching problem

Here's what actually happens when a team tries to build a multi-agent without designing it first: they get one agent working, then a second, then try to connect the two. The connection is where it falls apart.

The Researcher's output doesn't match the format the Drafter expects. Nobody defined who's responsible when the Drafter gets bad input — does it flag it, guess, or fail silently? There's no gate anywhere, so the first time something goes wrong in production, it's already gone wrong in production. Access wasn't scoped per-agent, so debugging which sub-agent touched what means reading raw logs line by line.

None of these are model problems. They're the predictable result of skipping the design step — role, capabilities, model, pipeline shape, and gates — and going straight to stitching prompts together. Two working agents connected without a design between them isn't a team. It's two single points of failure, wired in series.

What good design looks like

The output of designing a multi-agent properly isn't a working demo. It's a spec — something you could hand to another person, or another system, and have them understand exactly how the team is meant to operate:

• Every sub-agent has a defined role, prompt, capability set, and model
• The pipeline shape — what's ordered, what's concurrent — reflects real dependencies, not guesswork
• Gates sit at the points where a wrong or irreversible action would actually matter, each with a defined timeout and on-reject behaviour
• Access is scoped per sub-agent, not shared broadly across the team
• Reporting lines are explicit, so any output can be traced back to the sub-agent and the step that produced it

That's the anatomy. Role, model, capabilities, pipeline, gates, and reporting — six decisions, made deliberately, before anything runs. Everything that makes a multi-agent trustworthy comes from having answered these up front, not from picking a better model after the fact.

Where Envelope fits

This is the design step Envelope is built around. You describe the team you want in plain language, and Envelope turns that into the actual spec — sub-agents with roles and models assigned, a pipeline with the right steps ordered or concurrent, and gates placed where they belong — instead of leaving you to stitch prompts together and discover the gaps in production.

If you want to see the schema this maps to directly, it's in the docs. If you're weighing where gates specifically belong in a pipeline, that's covered in more depth in Human in the loop and the autonomous agent problem. And if model routing is the part you're trying to get right, The model path won't solve coordination goes further into why the model choice is only part of the job.

For a step-by-step walkthrough of the full design process, the multi-agent guide covers each decision in depth. If you're designing agents on top of specific platforms, there are worked examples for HubSpot, Salesforce, and Zendesk. If you're doing this work for clients, multi-agent systems and agencies covers how the delivery workflow adapts.

Design your AI agents in Envelope

Envelope turns a plain-language description of your workflow into a complete AI agent system — agents with named roles, model assignments, tool access, handoffs, and human review gates. Free to start, no code required.

Start designing →

Frequently asked questions

What is a multi-agent?

A multi-agent is a small team of specialised AI agents that work together on a task too broad for one agent to handle well. Each agent has a narrow role, its own model and tools, and a defined handoff to the agents around it.

What's the difference between a multi-agent and a sub-agent?

The multi-agent is the whole team. A sub-agent is one member of it — a single, narrowly defined role with its own prompt, capabilities, model, and reporting line.

Why not just use one agent with a long prompt?

One agent asked to handle a job with more than one kind of decision in it ends up mediocre at all of them — cautious where it should be fast, verbose where it should be terse. Specialised sub-agents, each doing one thing well, outperform a generalist doing several things adequately.

What is a gate in a multi-agent pipeline?

A gate is a checkpoint where the pipeline stops and waits for a human decision before continuing — not a notification or a log entry, but an actual pause before an irreversible or high-cost action.

Do all sub-agents need to use the same AI model?

No. Model routing means matching each sub-agent's model to what its role actually demands — a classification task might use a cheap, fast model, while a judgment-heavy escalation might need your strongest one.

Why do most attempts to build a multi-agent fall apart?

Most teams skip the design step and go straight to stitching prompts together: no defined roles, no gates, no scoped access. The failures show up at the connections between agents, not inside any single agent.

Can Envelope generate a multi-agent design automatically?

Yes. You describe the team you want in plain language, and Envelope turns that into a full spec — sub-agents with roles and models assigned, a pipeline with steps ordered or run concurrently, and gates placed where they belong.