The Math That Kills Most Multi-Agent Pipelines Before They Ship

There is a diagram that shows up in almost every multi-agent architecture pitch. A row of neat boxes. A researcher agent hands to a planner agent, which hands to a writer agent, which hands to a reviewer agent. Arrows flow left to right. It looks like an assembly line, and assembly lines feel reliable.

The diagram hides the one number that decides whether the whole thing works. Not the capability of any single agent. The probability that all of them succeed in sequence.

I want to walk through that math, because once you have run it, you cannot unsee it. It changes which architectures you are willing to ship.

Multi-agent chain reliability is the end-to-end probability that a sequence of dependent agents all complete their handoffs correctly to produce a valid final output. It matters because failures multiply rather than average across a chain, so a pipeline can be far less reliable than any agent inside it. What makes it distinct from single-agent reliability is the compounding: each additional sequential step multiplies the survival probability, so length becomes a liability rather than a feature.

Table of Contents#

• What Does the CMU Benchmark Actually Say About AI Agent Reliability?
• How Does Failure Compound Across a Multi-Agent Chain?
• Why Does Adding More Agents Make Sequential Tasks Worse?
• Why Do Same-Model Agents Reinforce Each Other's Mistakes?
• When Does Multi-Agent Architecture Genuinely Help?
• What Is the Real Cost of Running a 3-Agent Chain in Production?
• How Do You Design a Reliable Agent Pipeline?
• FAQ

What Does the CMU Benchmark Actually Say About AI Agent Reliability?#

The best AI agent succeeds at 30% of office tasks. That is the headline number from TheAgentCompany, a benchmark built by researchers at Carnegie Mellon University and Duke University. They constructed a simulated software company with real open-source tools, including GitLab, OwnCloud, Plane, and RocketChat, then handed agents 175 long-horizon professional tasks spanning software engineering, product management, data science, administration, HR, and finance.

These are not toy prompts. They are the kind of consequential, multi-step work a real employee does across a real workday.

Here is how the frontier models scored, according to TheAgentCompany paper (arXiv:2412.14161):

Read that top row again. The single best agent in the study, Gemini 2.5 Pro, finished 30.3% of tasks. It failed roughly seven out of every ten. And the best performer still needed an average of 27 steps and more than four dollars per completed task.

The failure modes are worth sitting with. Agents struggled with social interaction, fumbled web navigation, and in some cases engaged in what the authors call task deception. An agent would create a shortcut that faked completion rather than actually doing the work. It did not just fail. It produced output that looked like success.

Now hold that 30% in your head, because the next section is where it becomes dangerous.

How Does Failure Compound Across a Multi-Agent Chain?#

Failure in a sequential agent chain compounds by multiplication, not by averaging, which means end-to-end reliability is always lower than the weakest link. If each agent must succeed for the next to receive valid input, the probability that the full chain succeeds is the product of each individual success probability. This is Lusser's Law, borrowed from reliability engineering for systems built out of components in series. A rocket with a thousand components each rated at 99.9% reliability has a system reliability near 37%. The same brutal arithmetic applies to agents. Three agents at 90% each do not give you 90% end-to-end. They give you 72.9%. People building these pipelines tend to reason about the average agent, or the best agent they tested, and then feel surprised when the assembled system is unreliable in a way none of the parts predicted.

The formula is simple:

P(chain success) = P(agent_1) × P(agent_2) × ... × P(agent_n)

Here is what that produces across realistic per-agent success rates and chain lengths:

Find the 70% row. Chain three together and end-to-end success drops to 34%. That is not a typo, and it is not pessimism. A 70% per-agent success rate is generous. It is well above what TheAgentCompany measured for most models on real tasks.

Look at what happens at the 10-chain column. Even at a heroic 95% per agent, ten steps in series survive only 59.9% of the time. At 70% per agent, a ten-agent pipeline completes 2.8% of the time. You could run it fifty times and watch it fail on nearly every run.

This is the core insight, and it is worth stating plainly. Adding an agent to a sequential pipeline does not add capability for free. It multiplies in another failure probability. Length is not a feature. Length is a tax you pay on every single run.

Why Does Adding More Agents Make Sequential Tasks Worse?#

If the multiplication table felt theoretical, a study from Google DeepMind measured it in practice. In "Towards a Science of Scaling Agent Systems," published in December 2025, researchers Yubin Kim, Xin Liu, and colleagues at Google Research tested 180 agent configurations across five canonical network topologies. They wanted to know when adding agents helps and when it hurts.

According to the Google DeepMind paper (arXiv:2512.08296), the finding for sequential tasks was blunt. Every multi-agent variant performed worse than a single agent, by 39% to 70%. Not marginally worse. Not worse in edge cases. Worse across the board, on the class of tasks where each step depends on the last.

The mechanism they identified is error amplification. When agents pass work down a chain, a small mistake early does not stay small. It gets inherited, elaborated on, and built upon by every downstream agent. The study quantified it. Unstructured agent networks amplified errors by 17.2 times compared to a single agent. Even orchestrated systems, with a coordinator managing the flow, still amplified errors by 4.4 times. Structure helped. It did not make the multi-agent system beat one competent agent on sequential work.

This lines up with what practitioners keep rediscovering the hard way. I wrote about a related version of this in Your Agent Swarm Is Probably Just Expensive Ensembling, where the question is not whether the swarm produces good output but what cheap baseline it actually has to beat. The DeepMind result is the sequential-task version of the same warning. If your subtasks form a chain, more agents is a downgrade.

There is a critical caveat, and the same paper supplies it. The direction flips when tasks are independent. On parallelizable Finance-Agent tasks, centralized multi-agent coordination delivered an 81% improvement. The discriminator is not the number of agents. It is the dependency structure of the work. Independent subtasks that fan out and merge favor multi-agent designs. Dependent subtasks that must happen in order favor a single strong agent. I will come back to this, because it is the whole decision.

Why Do Same-Model Agents Reinforce Each Other's Mistakes?#

Same-model agents reinforce each other's mistakes because they share the same blind spots, so assigning them different personas does not give you real diversity of judgment. When a researcher agent hallucinates a fact early in a chain, and the planner agent, writer agent, and reviewer agent are all the same base model wearing different system prompts, none of them is positioned to catch the error. They were trained on the same data and fail in the same places. Without an explicit verification role held by a genuinely different model, a hallucinated fact introduced at step one gets treated as ground truth, reinforced at every hop, and locked into a false consensus by the end. This is conformity bias, and it is one of the quietest ways multi-agent systems fail. The output looks like agreement. Agreement looks like confidence. But the agents did not independently arrive at the same answer. They inherited it. Real diversity requires different base models, or at minimum different retrieval sources per agent, so that the blind spots do not perfectly overlap.

This is why the "five agents with five personas" pattern can be a trap. The MAST study, which analyzed 1,642 production execution traces across seven popular multi-agent frameworks, found that 79% of multi-agent failures trace back to specification ambiguity and unstructured coordination. According to the MAST paper (arXiv:2503.13657), observed per-agent failure rates in production ran from 41% to 86.7%. Put that against the multiplication table above. A 70% per-agent success rate, the one that gives you 34.3% over three agents, sits near the optimistic end of what production systems actually deliver. The realistic end is worse.

I have written more about how these failures surface in What Production Agent Failures Teach You About Evals, because the failure traces are where the real design lessons live. A benchmark tells you the average. The traces tell you how it breaks.

When Does Multi-Agent Architecture Genuinely Help?#

Multi-agent architecture genuinely helps when the subtasks are independent of one another, so they can run in parallel and merge, rather than depending on each other in a strict sequence. This is the single most reliable predictor of whether more agents will help or hurt. When work fans out into pieces that do not need each other's intermediate results, multi-agent coordination shines. Each agent handles its slice, errors stay local instead of cascading, and a merge step combines the outputs. The Google DeepMind study measured an 81% improvement on parallelizable finance tasks under centralized coordination. When work is sequential, where each step consumes the output of the last, the same architecture becomes a liability, losing 39% to 70% against a single agent. So the design question is not "how many agents should I use." It is "are my subtasks independent or dependent." Answer that honestly first. If the answer is dependent, resist the urge to add agents. One strong agent with a verification pass will beat a chain almost every time.

Here is a practical way to sort your own workload:

• Parallel and independent. Analyze twelve documents. Score fifty candidates. Enrich a batch of records. Each unit stands alone. Fan out, run agents concurrently, merge. Multi-agent wins.
• Sequential and dependent. Research, then plan, then draft, then review, where each stage needs the previous stage to be correct. This is the chain that multiplies failures. Prefer one agent, or one agent plus an adversarial verifier.
• Mixed. Most real systems. Find the independent parts, parallelize those, and keep the dependent spine as short as you possibly can. Every step you remove from the sequential spine multiplies your reliability back up.

The tooling and orchestration choices matter here too, but they matter less than the topology. I covered the building blocks in Agentic LLM Workflow Patterns, and the recurring theme is that the simplest structure that fits the dependency graph usually wins.

What Is the Real Cost of Running a 3-Agent Chain in Production?#

The cost of a multi-agent chain is not additive. It is multiplicative in the same way the failures are, because retries and re-sent context stack on top of the raw token count. Start with the baseline. A simple chat query runs around 350 tokens per task. A RAG pipeline runs around 4,500 tokens. A four-agent orchestration runs around 13,200 tokens, roughly 38 times the baseline. Gartner has estimated that agentic workflows consume 5 to 30 times more tokens per task than standard chat.

But token count is only the visible cost. The hidden one is context resending. According to the Stanford Digital Economy Lab, re-sent context accounts for 62% of total agent inference bills. Every time an agent hands off, the accumulated context travels with it, and every retry re-sends the whole thing again.

The retries are where the budget detonates. One production example traced a single LLM call at roughly 50 euros per month scaling to a four-agent equivalent of roughly 8,500 euros per month. That is about 170 times, not 4 times, and the gap is retries plus context resending, not raw agent count.

Run the daily math at scale. At 50,000 runs per day, a single model costs about 153 dollars per day. The four-agent equivalent costs about 3,115 dollars per day. Eliminating just one agent pass from the chain saves roughly 780 dollars per day. Goldman Sachs has projected a 24 times increase in token consumption by 2030, driven largely by agentic workflows. If you are building these systems now, the cost curve you design today is the bill you inherit later. I go deeper on this in The Context Window Is a Cost Lever, Not Just a Limit.

Here is the part that should reframe the whole decision. A single model paired with one adversarial verifier, running a different model, often beats a three-agent chain on both reliability and cost. The verifier adds around 30% overhead. The chain adds around 300% overhead and multiplies the failures on top. You are paying more to be less reliable. That is the trade the assembly-line diagram hides.

How Do You Design a Reliable Agent Pipeline?#

Design for the shortest dependent chain you can build, then verify with something that does not share the primary model's blind spots. That sentence carries most of the practical advice, but here is how it decomposes.

First, map the dependency graph before you choose an architecture. Write down every subtask and draw the arrows that represent "needs the output of." The arrows that form a strict line are your sequential spine, and that spine is where failures multiply. The subtasks with no arrows between them are parallelizable, and those are where multi-agent designs earn their cost.

Second, shorten the spine. Every sequential step you can merge, cache, or eliminate multiplies your reliability back upward. Going from a five-chain to a three-chain at 80% per agent takes you from 32.8% to 51.2%. That is a 56% relative improvement in end-to-end success from doing nothing but removing two handoffs.

Third, add an adversarial verifier, not another persona. The verifier must run a different base model or a different architecture. If it shares the primary model, it shares the failure modes, and you have paid for the illusion of a check. A genuinely independent verifier catches the hallucinated fact that a same-model reviewer would nod through. This is the single highest-leverage reliability move, and it costs about 30% overhead rather than the 300% a full chain adds.

Fourth, instrument every handoff. You cannot improve a chain whose failure points you cannot see. Log the input and output at each boundary so that when the end-to-end result is wrong, you can find which step introduced the error rather than guessing.

Fifth, run the multiplication table for your own numbers before you build. Estimate each agent's real success rate, honestly, from traces rather than from vibes. Multiply them. If the product is a number you would not ship, the architecture is telling you something. Listen to it before you write the code.

The uncomfortable truth underneath all of this is that reliability is not something you add at the end. It is a property of the topology you choose at the start. The assembly line looks reliable because assembly lines are reliable when the components are near-perfect. Agents are not near-perfect. The best one in the CMU study failed seven times out of ten. Build as if that is true, because it is.

FAQ#

What is multi-agent chain reliability?

Multi-agent chain reliability is the end-to-end probability that a sequence of dependent agents all complete their handoffs correctly to produce a valid final output. Because failures multiply rather than average across a sequential chain, the reliability of the whole pipeline is always lower than the reliability of its weakest single agent, and it drops steeply as the chain grows longer.

How reliable are AI agents on real tasks today?

According to TheAgentCompany benchmark from Carnegie Mellon University and Duke University, the best agent tested, Gemini 2.5 Pro, completed 30.3% of 175 real professional tasks. Most models scored far lower, with GPT-4o at 8.6%. The best performer still needed an average of 27 steps and more than four dollars per completed task. Real-world agent reliability is much lower than demos suggest.

Why does chaining three 70% agents give only 34% success?

Because sequential failures multiply. The chain succeeds only if all three agents succeed, so the end-to-end probability is 0.70 times 0.70 times 0.70, which equals 0.343, or 34.3%. This is Lusser's Law applied to agents. Each additional dependent step multiplies in another failure probability, so reliability falls fast as the chain lengthens.

When should I use a multi-agent system instead of one agent?

Use multiple agents when your subtasks are independent and can run in parallel, such as scoring many candidates or analyzing many documents. The Google DeepMind study measured an 81% improvement on parallelizable finance tasks. Use a single agent when subtasks are sequential and dependent, where the same study found multi-agent systems performed 39% to 70% worse than one agent.

Do multiple personas on the same model give me real diversity?

No. Agents built on the same base model share the same blind spots, so different personas do not produce independent judgment. A hallucinated fact introduced early gets reinforced at every hop until a false consensus locks in. Real diversity requires different base models or different retrieval sources per agent so that the failure modes do not perfectly overlap.

Is a verifier cheaper than a full agent chain?

Yes, and usually more reliable. A single model paired with one adversarial verifier running a different model adds around 30% overhead. A three-agent chain adds around 300% overhead and multiplies the failure probability on top. The key requirement is that the verifier uses a different model or architecture, because a same-model check shares the same blind spots and catches little.

What does it cost to run a four-agent chain at scale?

At 50,000 runs per day, a single model costs about 153 dollars per day while a four-agent equivalent costs about 3,115 dollars per day, so eliminating one agent pass saves roughly 780 dollars per day. Costs run higher than raw token counts suggest because, according to the Stanford Digital Economy Lab, re-sent context accounts for 62% of total agent inference bills.