Building a Multi-Agent Research System: Patterns from Production

When a single-agent system fails at a research task, the instinct is to add more memory, better tools, or a smarter model. But there's a point where the problem isn't capability — it's concurrency. Deep research tasks require pursuing multiple threads simultaneously: validating claims from different angles, scanning sources across domains, cross-referencing findings in real time. A single agent doing this sequentially is like a researcher reading every book one at a time before taking notes. The multi-agent alternative feels obvious in retrospect, but getting it right in production is considerably harder than the architecture diagram suggests.

This post is about how multi-agent research systems actually get built — the architectural choices that work, the failure modes that aren't obvious until you're in production, and the engineering discipline required to keep them useful at scale.

The Core Pattern: Orchestrator with Parallel Subagents

The architecture that works for deep research tasks is a lead orchestrator that delegates to parallel subagents. When a query arrives, the orchestrator analyzes the request, decomposes it into independent subtasks, and spawns 3–5 subagents to explore different aspects concurrently. A synthesis step collects and reconciles the subagent outputs. An additional citation agent handles source attribution separately, keeping that concern isolated from the research logic.

This design outperforms single-agent approaches by a substantial margin on complex queries — particularly those that benefit from breadth-first exploration. The performance gain comes from two sources: parallelism (subagents work simultaneously rather than sequentially) and specialization (each subagent receives a focused, bounded objective rather than the full sprawling query).

The orchestrator's job is not to do research — it's to design the research strategy. That distinction matters when you're writing prompts. An orchestrator prompt that bleeds into retrieval details produces agents that neither orchestrate nor retrieve well.

What Goes Wrong Without Discipline

The failure modes in multi-agent research systems are consistently surprising to engineers who've only worked with single-agent loops. Here are the ones that appear repeatedly in production:

Agent sprawl. Without explicit guidance, an orchestrator will spawn the same number of subagents for a simple factual query as for a deep comparative analysis. Spawning 50 agents to answer a question that needs one is not just expensive — it produces worse results because the synthesis step becomes noise reduction rather than insight generation. The fix is explicit spawn guidance in the orchestrator prompt: single-agent for fact lookups, 2–3 for moderately complex tasks, 10+ only for tasks that genuinely require parallel investigation.

Vague delegation. "Research topic X" is not a subagent instruction. Without distinct, non-overlapping objectives, subagents duplicate work or leave gaps. The orchestrator needs to provide each subagent with a specific scope: what to look for, what format to return, what tools to use, and — critically — what the other subagents are already handling. The last point prevents redundancy.

Search strategy collapse. Left to their own devices, agents converge on the same high-visibility sources that dominate search results. SEO-optimized content crowds out authoritative primary sources. Starting broad and progressively narrowing scope — rather than immediately going narrow — produces more diverse and reliable retrieval.

Cascading errors from statefulness. In long-running research sessions, errors compound. A subagent that misinterprets an early intermediate result can send an entire thread in the wrong direction. The practical mitigation is checkpointing: agents that can resume from known-good states rather than restart from scratch when something fails. This also matters operationally — research tasks running for 20–30 minutes cannot afford cold restarts.

Synchronous bottlenecks. Waiting for all subagents to complete before the orchestrator can proceed is the simplest coordination model, and it works. But it creates latency spikes when one subagent hits a slow source and blocks the rest. Asynchronous execution patterns address this but introduce their own coordination complexity around partial synthesis and dynamic delegation.

Token Economics Are Not Optional

Multi-agent systems are expensive. A typical research session consuming 4× the tokens of a chat interaction is a rough rule of thumb; production data shows the range is wide and skewed. Three variables explain most of the cost variance: token usage, tool call count, and model selection. These aren't independent — they interact.

The practical approach is tiered model allocation. Use a capable frontier model for the orchestrator, which requires strategic reasoning and synthesis. Use a faster, cheaper model for subagents doing retrieval and summarization. This isn't a quality compromise if the task decomposition is clean — subagents performing bounded retrieval tasks don't need the same reasoning depth as the orchestrator synthesizing their outputs.

Caching matters significantly for research systems because many subagent calls share overlapping context: the system prompt, the original query, intermediate summaries. Prompt caching on the static portions of these can reduce costs materially on repeated or similar queries.

The economic case for multi-agent research systems rests on task value, not architectural elegance. For a query where finding a critical source saves significant time or avoids a costly mistake, the token spend is justified. For routine lookups, it isn't. Building that routing logic — to single agent vs. multi-agent — is itself a piece of system design that teams frequently skip, and then wonder why their costs are uncontrolled.

Evaluation Is the Lever That Actually Moves Quality

Because multi-agent systems are non-deterministic, evaluation is harder than for single inference calls. The same query can produce different subagent breakdowns, tool sequences, and synthesis results across runs. Traditional unit test approaches don't generalize well here.

What works: start with a small representative query set, around 20 queries that cover the distribution of task complexity your system will actually see. LLM-as-judge evaluation, with a single prompt scoring factual accuracy, citation precision, completeness, source quality, and tool efficiency on a 0–1 scale, correlates well with human judgment and scales to the volume needed for iteration.

Human evaluation remains irreplaceable for catching the failure modes that automated scoring misses: selection bias toward easily-findable but low-quality sources, systematic gaps in domain coverage, and edge cases that score well on the rubric but frustrate users. Automated eval and human review are complements, not substitutes.

The most important operational point: start evaluating early. A 20-query eval suite run against early architecture decisions surfaces problems before they're baked into the system. Teams that wait for a comprehensive benchmark before evaluating consistently lose weeks to preventable regressions.

Context at Scale: The 200-Turn Problem

A research agent managing 200+ conversation turns faces a structural problem: the context window cannot hold the full session, and attention degrades on long contexts even when it technically fits. The solution requires treating memory as an explicit system concern rather than an implicit side effect of the context window.

The pattern that works: agents store plans, intermediate summaries, and key findings in external memory stores at regular intervals. When subagents are spawned, they receive a fresh context initialized from the relevant subset of stored memory rather than the full conversation history. This keeps each agent's context window focused and prevents the dilution that comes from carrying irrelevant history across hundreds of turns.

Handoffs between agents — the moment one agent's work becomes another agent's input — are the highest-risk points for information loss. Explicit handoff protocols, including what information must be passed, in what format, and with what level of certainty indicated, prevent the silent failures where agents proceed confidently on misunderstood premises.

Production Operations: Deployment and Debugging

Deploying multi-agent systems requires rethinking several standard practices. Rolling deployments that cut over all traffic simultaneously will interrupt in-flight research sessions — some of which may be running for 20–30 minutes. The alternative is gradual traffic shifting between versions, keeping the old version alive long enough for running sessions to complete before draining it.

Debugging non-deterministic agents requires tracing that captures decision structure without exposing conversation content. The useful signal is the interaction graph: which agents were spawned, in what order, with what tool call sequences, and where coordination broke down. Standard application monitoring tools don't surface this by default.

The monitoring question that matters most in production is not "did this request fail?" but "is this agent making good decisions?" That requires sampling task outputs and evaluating them against rubrics, not just tracking error rates and latency percentiles.

When Not to Use Multiple Agents

Multi-agent research systems are the right choice for tasks requiring genuine breadth-first exploration: questions with multiple independent dimensions, tasks requiring cross-validation of claims across sources, research that benefits from parallel hypothesis testing. For single-answer factual queries, document summarization, or anything where the answer space is narrow, a single well-prompted agent is faster, cheaper, and more reliable.

The tendency to reach for multi-agent architecture as a default — rather than as a tool for a specific class of problem — produces systems that are hard to debug, expensive to run, and slower than simpler alternatives. The discipline is knowing when the complexity is earned.

What earns that complexity: tasks where finding a critical piece of information in parallel saves meaningful time, tasks where cross-referencing prevents costly errors, and tasks where users will actually use the full depth of research rather than stopping at the first adequate answer. If you can't articulate why parallelism helps for your specific use case, you probably don't need it.