The Telephone Game in Your Agent Pipeline

Here is a failure you have probably watched happen without naming it. Your orchestrator reads the user's request and briefs a worker agent. The worker runs a dozen tool calls, digests the outputs, and reports back a tidy summary. The orchestrator folds that summary into a brief for the next worker, who does the same. Five hops later, the system delivers a confident final answer — and it violates a constraint the user stated plainly in the second sentence of their request. Nobody dropped the constraint on purpose. Each hop just compressed the context a little, in a direction nobody chose, and the compressions compounded.

This is the telephone game, and multi-agent systems play it constitutively. Every handoff in your pipeline is a lossy compression step: the orchestrator's brief is a paraphrase of the user, the worker's report is a paraphrase of its tool outputs, and the final answer is a paraphrase of paraphrases. The question is not whether information gets lost — it does, measurably — but whether you have decided what is allowed to be lost, or left that decision to the sampling temperature.

The failure data backs this up. The MAST taxonomy — built from over 1,600 annotated execution traces across seven popular multi-agent frameworks — found that inter-agent misalignment accounts for roughly 37% of all multi-agent failures: communication breakdowns, context lost during handoffs, conflicting outputs, format mismatches between agents. That is not a tail risk. It is the second-largest failure category in the taxonomy, and unlike model capability, it is entirely an artifact of how you wired the system together.

Every Hop Is a Compressor Nobody Tuned

When an agent summarizes context before a handoff, it is running a compression algorithm whose loss function you never specified. The agent decides — implicitly, token by token — what is salient enough to keep. And its notion of salience is generic: trained on what usually matters in text, not on what matters in your pipeline.

Research on iterated LLM generation makes the compounding explicit. The "broken telephone" studies, which chain models in transmission loops the way humans once chained whispered sentences, found that distortion accumulates steadily across iterations and drifts toward attractors — texts get shorter, more generic, more positive, and converge on whatever phrasing the model finds most natural. A bias invisible in any single generation becomes structural after five. Your pipeline is a transmission chain with extra steps.

The loss is not random noise, which would almost be easier to handle. It has a characteristic shape:

• Qualifiers die first. A study of LLM-compressed financial analysis identified decontextualization as a dominant failure pattern: the salient fact survives the summary, but the conditions attached to it do not. "Revenue grew 40% (driven by a one-time contract)" becomes "revenue grew 40%." The downstream agent then reasons correctly from a premise that is now wrong.
• Groundedness collapses under compression. Work on prompt compression found that aggressive compression drops groundedness scores by 30 to 50 points — the compressed context still reads fine, but downstream generations are dramatically less faithful to the original source.
• Negative results evaporate. A worker that tried three approaches and found two dead ends reports the one that worked. The dead ends — which would have saved the next agent from repeating them — rarely survive the summary, because summaries are biased toward conclusions over process.
• Implicit decisions vanish entirely. As Cognition's engineering team put it: actions carry implicit decisions, and conflicting decisions carry bad results. When a worker chose library A over library B, that choice constrains everything downstream. But it was never stated as a decision, so no summary preserves it.

Notice what all four have in common: the summary is not wrong. It is accurate at the resolution it chose. The damage comes from the resolution.

Why You Don't Notice Until Hop Five

Single-hop loss is small, which is exactly why it survives code review. You inspect one handoff, the summary looks faithful, you ship. But loss per hop is multiplicative, not additive. If each hop preserves 90% of decision-relevant information — a generous number, given the groundedness data — a five-hop chain delivers 59%. At 80% per hop, you are at 33%.

The confidence signal is worse than useless here, because it is anti-correlated with fidelity. Each agent in the chain writes fluent, assured prose about the context it received. The final answer arrives with the same polish whether it is built on the user's actual constraints or on a fifth-generation paraphrase of them. Fluency is the great masking agent of the telephone game: in human chains, garbled messages sound garbled; in LLM chains, every retelling is impeccably written.

This also explains a pattern many teams hit: the multi-agent version of a system underperforms the single-agent version on tasks that fit in one context window. The MAST authors note that multi-agent performance gains over single-agent baselines are often minimal on popular benchmarks. One reason is that you pay the telephone tax on every handoff, and if the task didn't need decomposition, the tax buys you nothing.

Anthropic's team, describing their research system, was explicit that the architecture wins on breadth-first problems where the total information genuinely exceeds one context window — that is when parallel subagents with separate contexts beat one agent, and their evals showed token budget explained the bulk of the performance variance. Handoffs are worth paying for when they buy you tokens you couldn't otherwise have. They are pure loss when they don't.

Measure the Loss Before You Fix It

You cannot tune a compressor without a loss function, so the first move is measurement, not redesign. Three practical probes:

Constraint tracing. Enumerate the explicit constraints in the original request — budget limits, format requirements, exclusions, deadlines — and grep for them (or their semantic equivalents) in the context each downstream agent actually received. This is tedious to do by hand and trivial to automate with an LLM judge: "Here is the original constraint list; here is what agent N received; which constraints are recoverable?" Plotting constraint survival per hop gives you a decay curve. Most teams who run this for the first time find at least one constraint that dies at hop two.

Round-trip probing. Borrowed from the compression literature: after a handoff, ask the receiving agent questions whose answers were in the pre-handoff context. If the orchestrator knew the user's API version and the worker cannot answer "which API version are we targeting," the handoff dropped it. This catches decontextualization that constraint lists miss, because you can probe for qualifiers, not just facts.

Decision archaeology on failures. When the pipeline produces a bad output, trace backwards: at which hop did the information needed to avoid the failure last exist? In my experience this is the highest-leverage debugging question in multi-agent systems, and most teams cannot answer it because they log agent outputs but not the assembled context each agent actually saw. Log the inputs. The outputs you can regenerate; the context assembly is the evidence.

These measurements usually reveal that loss is concentrated, not uniform — one or two handoffs in the pipeline do most of the damage, typically the ones where a general-purpose "summarize your findings" instruction stands in for an actual specification of what downstream needs.

Verbatim Lanes and Compression Lanes

Once you can see the loss, the design question becomes: what must travel verbatim, and what can be compressed? Treating this as a per-field decision, rather than a per-message one, is the core move.

Some things should never pass through a paraphrase:

• User-stated constraints and acceptance criteria. These are the ground truth the whole pipeline serves. Copy them, don't summarize them. They are usually short; the token cost is trivial against the failure cost.
• Identifiers and exact values. File paths, API versions, error messages, numbers, IDs. LLMs are notorious for "normalizing" these in retelling — the broken-telephone drift toward the plausible is deadliest on strings where plausible and correct diverge.
• Decisions with rationale. Not "I set up the database" but "chose Postgres over SQLite because we need concurrent writers." The rationale is what prevents a downstream agent from silently reversing the decision.
• Negative knowledge. Approaches tried and failed, options ruled out and why. This is the information most reliably destroyed by summarization and most expensive to rediscover.

Everything else — exploration narration, raw tool output that has been distilled into a decision, the worker's reasoning transcript — can be compressed, provided the original stays retrievable. This is why artifact-passing beats summary-passing as a default: the worker writes its full output to a shared store (a file, a blob, a row) and hands off a reference plus a summary. The summary is now an index into the truth rather than a replacement for it. The receiving agent reads the summary, and when something looks load-bearing, it pulls the artifact. Compression becomes a performance optimization instead of an information ceiling — the same reason we build caches over databases rather than caches instead of databases.

Anthropic's research system does a version of this: subagents write findings to persistent storage and pass lightweight references, precisely so the orchestrator's synthesis isn't limited to what fits in a handoff message. Cognition goes further and argues for sharing full agent traces, not summaries at all — which is the same principle taken to its limit, viable when your hops are few and your context budget generous.

The Fix Is a Schema, Not a Better Prompt

The instinctive fix is prompt engineering: "make sure to include all important constraints in your summary." This fails for a structural reason — it delegates the compression decision back to the model's generic salience, just with more emphatic phrasing. You are asking the compressor to please be less lossy, without telling it what loss means for you.

The durable fix looks like schema design. Define the handoff as a typed contract: a structured object with explicit fields — constraints (copied verbatim from origin), decisions (each with rationale), artifacts (references to full outputs), open_questions, ruled_out, and a free-text summary for everything genuinely compressible. Research on inter-agent communication supports the intuition: studies comparing handoff strategies find that what you pass — the artifact, the conclusion, the structured state — changes downstream task performance far more than how eloquently the passing message is phrased.

The schema does three things a prompt cannot:

1. It makes omission visible. An empty ruled_out field is a reviewable event; a summary that happens not to mention dead ends is invisible. Required fields turn silent loss into a validation error.
2. It separates lanes mechanically. Verbatim fields are populated by copying, not generation — the constraint text that reaches hop five is byte-identical to hop one, because no model ever rewrote it. You cannot paraphrase what you are forbidden to touch.
3. It versions like an API. When downstream agents need new information, you extend the contract and update producers — the same discipline as any interface change, testable in isolation. Prompts have no equivalent of a breaking change; schemas do.

This is also why the problem feels familiar to anyone who has built distributed systems. Services don't exchange freeform prose about their state; they exchange schemas, because we learned decades ago that implicit contracts between components drift until they fail. Multi-agent pipelines are distributed systems whose messages happen to be written by a stochastic process. That makes explicit contracts more necessary, not less.

Fewer Hops, Fatter Pipes

Zoom out and the telephone game suggests an architectural heuristic: every handoff must pay for itself. A hop is justified when it buys parallelism across genuinely independent subtasks, or context capacity the task actually needs, or isolation you can defend. A hop that exists because the org chart of your agents mirrors the org chart of your team — a "planner" that paraphrases the user for the "researcher" who paraphrases sources for the "writer" — is a compression stage with no compensating gain.

So before adding an agent, ask what the handoff will lose and what it will buy. Prefer fewer, fatter hops: two agents exchanging rich structured state beat five exchanging elegant summaries. Give every hop a schema with verbatim lanes for constraints, identifiers, decisions, and dead ends. Store full outputs as artifacts and pass references. And measure constraint survival across your pipeline the way you measure latency across your services — because right now, the message your fifth agent is acting on is not the message your user sent, and no one in the chain can tell you what changed.