Tiago Sampaio

main ai engineering career

ai

8 min read

The 5 Ways to Organize a Team of AI Agents (and How to Pick the Right One)

Pipeline, orchestrator-worker, swarm, mesh and hierarchical: how each AI agent pattern works, when it fits, what it costs, and a simple rule to pick one.

ai-agent-orchestration-patterns.png — 1920 × 1080

Cover: "More agents won't make AI smarter. Organization will." Orchestrator-worker, pipeline, swarm, mesh and hierarchical. Panel: tokens used vs. a chat, about 4x for a single agent and about 15x for a multi-agent system (Anthropic Engineering, June 2025).

If you had five AI agents working for you, who would give the orders? Most people skip that question and go straight to adding more agents. That is how a system ends up costing many times more than a single chat without giving a better answer. Anthropic, the company behind Claude, measured it: its multi-agent systems use about 15 times more tokens than a normal chat.

More agents don’t make AI smarter. Organization does. I lead a development team that builds Magento and Adobe Commerce stores for clients in the US, and the questions are the same ones I answer with people every week: who does what, in what order, and how the work moves from one hand to the next.

In this article I go through the five most common ways to organize a team of agents, from the simplest to the most complex. For each one: how it works, when it fits and what it costs. At the end I give you the rule I use to pick one without burning money.

From one agent to a team: why orchestration matters

An AI agent is a language model, like Claude or ChatGPT, that does more than answer. It plans steps, uses tools (search the web, read files, run code) and decides what to do next on its own.

One agent with one clear task usually does a good job. The trouble starts when you pile everything on the same agent: research the market, compare five vendors, write the report, check the numbers, format the slides. Its context fills up, which is the amount of text the model can keep in view at once. Instructions start competing with each other, and the agent loses the thread.

The natural move is to split the work among specialists. One agent searches, another summarizes, another writes. That works, but it creates a new problem. Someone has to decide how those agents work together. That decision has a name: orchestration.

Orchestrating agents sounds complicated, but it is the same thing a tech lead does with a team. Every orchestration answers three questions:

  1. Who does what. Which agent owns each part of the task.
  2. In what order. Does one wait for the other, or do they work at the same time?
  3. How the work is handed over. What exactly goes from one agent to the next: the full text, a summary, a list of findings?

Take a simple research flow: search, summarize, answer. Change the answers to those three questions and you get a completely different system, with a different cost, speed and failure mode. The five patterns below are five standard sets of answers.

One note on names before we start. There is no official standard. The names below are the ones you will see most often in articles, frameworks and vendor pitches, but they vary. Anthropic itself, in its guide Building effective agents, uses terms like “prompt chaining” and “orchestrator-workers”. What matters is the shape of the team.

Pipeline: the assembly line

Code
Search → Summarize → Write → Review

How it works. Each agent does one step and passes its output to the next. Agent two starts only when agent one finishes. Anthropic calls the basic version of this “prompt chaining”: a task broken into “a sequence of steps, where each LLM call processes the output of the previous one.”

When it fits. When the steps are fixed and always happen in the same order. Think of turning a meeting transcript into notes, then into tasks, then into an email. It is predictable, and debugging is easy: you look at the output of each step and find where it went wrong. You can also add a check between steps, a “gate” in Anthropic’s words, to stop the line before a bad result moves on.

The cost. An error early in the line flows all the way down. If the search step brings the wrong data, the summary, the text and the review will all be built on it. And it is slow by design, because nothing runs in parallel.

Orchestrator-worker: the boss and the team

Code
            Orchestrator
          /      |      \
     Worker   Worker   Worker
          \      |      /
            Orchestrator (joins the results)

How it works. One agent, the orchestrator, receives the task, splits it into parts and sends each part to a worker agent. The workers do their part, often at the same time, and the orchestrator puts the results together.

A real example. Claude’s Research feature works like this. In Anthropic’s write-up on how they built it, the lead agent “analyzes it, develops a strategy, and spawns subagents to explore different aspects simultaneously.” Anthropic reports that a multi-agent setup with Claude Opus 4 as the lead and Claude Sonnet 4 subagents outperformed a single Claude Opus 4 agent by 90.2% on its internal research evaluation.

When it fits. When the task splits into independent parts and you can’t know those parts in advance. Anthropic’s guide describes orchestrator-workers as a fit for tasks “where you can’t predict the subtasks needed.”

The cost. The orchestrator is a single point of failure. If it splits the task badly, every worker does the wrong job well. Anthropic also admits that early versions of its research agents made mistakes like “spawning 50 subagents for simple queries.”

Swarm: no boss

Code
Agent A ⇄ Agent B
⇅ ⇅
Agent C ⇄ Agent D

How it works. There is no central coordinator. Each agent decides when to act and when to pass the task to another agent better suited for the next part. Think of a support desk where whoever picks up the call transfers it to the right specialist.

When it fits. Open-ended flows where you can’t draw the path in advance, like a customer conversation that can go to billing, then to technical support, then back.

The cost. It is flexible, and that flexibility is the risk. The same input can take a different path every time, which makes it hard to predict, to test and to explain why the system did what it did. Without good limits, agents can pass the task around in circles.

Mesh: everybody in the same group chat

Code
   A --- B
   | \ / |
   | / \ |
   C --- D

How it works. Every agent can talk to every other agent. Picture a WhatsApp group where anyone can reply to anyone.

When it fits. Debate and review. One agent proposes, others criticize from different angles (security, cost, user experience), and the discussion improves the answer.

The cost. The number of connections grows fast. With n agents there are n × (n − 1) / 2 possible pairs. Four agents means 6 connections. Six agents means 15. Every connection is a conversation that consumes tokens and a place where a misunderstanding can start. The group chat that worked with three people becomes noise with ten.

Hierarchical: the company org chart

Code
                 Director
            /                \
     Manager A            Manager B
     /      \              /      \
 Worker   Worker       Worker   Worker

How it works. A top agent splits the problem into large areas and hands each one to a manager agent. Each manager runs its own team of workers. It is orchestrator-worker stacked in layers.

When it fits. Problems too big for one boss. When a single orchestrator would have to track dozens of subtasks, its own context becomes the bottleneck. Adding a middle layer keeps each agent’s view small.

The cost. Every layer adds latency, tokens and one more place where information gets lost in the handover. It is the same problem as in a real company: the message from the director reaches the team after three summaries, and part of it is gone.

The detail that separates saving money from burning it

Here is the part most diagrams leave out. Multi-agent systems are expensive. In the same research write-up, Anthropic says agents “typically use about 4× more tokens than chat interactions,” and multi-agent systems “use about 15× more tokens than chats.” Tokens are the pieces of text the model reads and writes, and they are what you pay for.

Anthropic is direct about what that means: multi-agent systems “require tasks where the value of the task is high enough to pay for the increased performance.” They also say where it does not work well today: domains “that require all agents to share the same context or involve many dependencies between agents.” The same write-up notes that “most coding tasks involve fewer truly parallelizable tasks than research.”

On the other side, Anthropic says these systems shine at “heavy parallelization, information that exceeds single context windows, and interfacing with numerous complex tools.”

So the cost is a reason to choose the pattern carefully. A wrong pattern rarely fails loudly. It quietly costs more and gives you a weaker result.

My rule: start with the simplest pattern that works

My take, after years of organizing real teams: you don’t hire a director before you have work for two people. Agents are the same. Anthropic’s own advice points the same way: find “the simplest solution possible, and only increasing complexity when needed.”

Think of the five patterns as a ladder, from 1 to 5, with a single agent below the first step. Start at the bottom and climb one step only when the step below can’t do the job. Here is the checklist I use, in order. Stop at the first “yes”:

  1. Can one agent with a good prompt and the right tools do it? Use one agent. In my experience, many tasks end here.
  2. Are the steps fixed and always in the same order? Use a pipeline.
  3. Does the task split into independent parts you can’t predict in advance? Use orchestrator-worker.
  4. Does the path depend on what happens along the way, with clear hand-offs between specialists? Consider a swarm, with strict limits on how many times a task can be passed.
  5. Do you need agents to critique each other’s work? Use a small mesh. Three or four agents, not ten.
  6. Is it too big for one orchestrator to track? Only then go hierarchical.

And before any of them, one question: does the value of the result pay for something like 15 times the tokens of a chat? If the answer is no, go back to step 1.

The next time someone in a meeting says “let’s throw a few AI agents at this,” you will know which question to ask: fine, but in which pattern?

Which of these patterns are you using today, and is it the simplest one that would work? Tell me in the comments. If you prefer video, I explain the same five patterns with drawings on my Instagram, @tiagoosampaio (in Portuguese).

Sources

Written by

TS

Tiago Sampaio

Adobe Commerce & Magento 2 Expert

Senior developer with 15+ years building software. Writes practical fixes for Magento 2 and Adobe Commerce, and honest notes on growing as a developer.

$ git remote add inbox

Get new commits by email

New posts from both branches, engineering and career. No spam, unsubscribe any time.

$ git notes show

Leave a Reply