Multi-agent systems in production: the parts nobody demos

Executive summary

A multi-agent system splits one big job across several specialized AI agents that talk to each other under a coordinator. In demos this looks effortless: three or four agents pass messages around, a polished result comes back, everyone nods. Production is different. Real traffic exposes race conditions, runaway costs, silent failures, and state that vanishes between steps.

This post covers what changes when multi-agent systems leave the demo stage. We'll look at why single agents hit a ceiling, how orchestrators coordinate specialists, where the Belief-Desire-Intention reasoning loop fits into modern LLM agents, which implementation steps take you from prototype to production, and what we learned running our own content pipeline this way.

The short version: these systems earn their keep in production when each agent has one clear job, the orchestrator owns all shared state, and every handoff has a timeout plus a fallback. Everything else is detail, and there's plenty of it below.

Why single agents stop scaling

A single agent with a long to-do list works until it doesn't. Context windows fill up, and the model starts ignoring constraints it agreed to ten steps earlier. One prompt ends up carrying tool selection, data formatting, quality checks, and error recovery at the same time, so a weakness anywhere drags down everything.

There's also a debugging problem. When one monolithic agent produces a wrong answer, you can't tell which responsibility failed. Was retrieval bad, or did the planner misread perfectly good retrieval? Teams end up staring at giant traces, trying to reverse-engineer their own system's logic after the fact.

Cost behaves badly too. Every retry re-runs the entire monolithic prompt, so a flaky tool at step nine makes steps one through eight billable again. Specialized agents isolate that blast radius: you re-run the broken part, nothing else.

Splitting work across specialists fixes all three problems. A research agent retrieves. A writing agent turns findings into a draft, and a reviewer checks every claim against sources before it goes anywhere. Each agent has a short context, a narrow job, and a testable output. That structure is what makes the rest of this post possible.

Technical deep dive: the orchestration layer

Every production multi-agent setup needs a component that decides who runs next, what they receive, and what happens when they fail. That component is the orchestrator, and its design matters more than your choice of models.

Orchestrator patterns: central vs. market

Central orchestration puts one coordinator in charge. It holds the plan, routes tasks, tracks progress, and owns retries. You get predictable behavior and one obvious place to look when things break. Most teams should start here and stay here longer than their architecture diagram suggests.

Market-style coordination lets agents bid on or claim tasks without a central boss. It absorbs flexible workloads nicely, but it's much harder to reason about, and failures surface as strange emergent behavior instead of clean errors. Treat it as an optimization for later, not a starting point.

Hybrid designs are common in practice. A central orchestrator handles planning while individual agents choose their own tools inside their lane. Global flow stays predictable, local execution stays flexible, and both properties are testable on their own.

Agent roles and the BDI reasoning loop

Each agent needs a defined role, its own context window, and a principled way to decide what to do next. The oldest formal answer is still the most useful mental model: Belief-Desire-Intention, drawn in the diagram above. An agent holds beliefs about the world, generates options from those beliefs, filters competing desires down to committed intentions, then acts and observes the result.

Modern LLM agents implement a version of this loop whether they name it or not. System prompts encode beliefs, task descriptions set desires, planners emit intentions, tool calls act on the world, and tool results update the beliefs again. Naming the loop makes failures legible: did the agent hold stale beliefs, or did it commit to an intention built on a belief it never verified? Those need different fixes.

At system scale, the same loop shows up twice. Each agent runs its own belief-to-intention cycle, and the orchestrator runs one too: agent outputs become its beliefs, and route changes become its intentions. Teams that skip that second layer end up with coordinators that hardcode every transition, which holds up until the first unexpected state arrives and nobody owns the decision about what happens next.

Communication, state, and memory

Agents need to exchange results without stepping on each other. Shared blackboards, message queues, and structured JSON handoffs all work well; free-form chat transcripts between agents do not scale and hide bugs beautifully. Whichever channel you pick, make the payload schema strict enough to validate automatically before the receiving agent spends a single token on it.

State belongs in a database, never in someone's conversation history. If an orchestrator crashes mid-run, a fresh process must reconstruct exactly where things stood from persisted records. Memory that lives only in prompts dies with the process, and it takes your user's half-finished job along with it.

Failure handling and observability

These systems fail in boring ways: a tool returns junk, an API times out, a model emits confident nonsense. Plan for each. Retries with exponential backoff absorb transient errors, circuit breakers keep one flaky dependency from burning the whole token budget, and anything that fails repeatedly lands in a dead-letter queue for a human to inspect.

Observability ties it together. Give every run an ID, log every inter-agent message with sender, receiver, latency, and token count, and track success rate per agent rather than only per pipeline. When an agent drifts after a model update, a per-agent dashboard surfaces the regression hours before your users would.

Implementation guide: from prototype to production

Here's the sequence we'd recommend to any team moving a multi-agent prototype toward production. It's deliberately boring, because boring is what reliable looks like.

• Give every agent exactly one responsibility and a written contract describing its input and output.

• Pick a central orchestrator first. Add fancier coordination later if measurements demand it.

• Persist all shared state externally. Agents read and write records; they never carry state in conversation.

• Budget each agent: maximum tokens, maximum tool calls, maximum wall-clock seconds. Enforce limits in code, not in prompts.

• Wrap every handoff in a timeout plus a fallback path, even if the fallback is a polite failure message to the user.

• Log every inter-agent message with correlation IDs so any run can be replayed step by step after an incident.

• Evaluate agents individually before evaluating the pipeline. A weak reviewer poisons everything downstream of it.

Roll out gradually once contracts exist. Run the new pipeline in shadow mode beside the old one for a week, compare outputs on identical inputs, and route real traffic across only when quality metrics match. Canary a small share of runs first, keep the rollback switch within reach, and leave the old pipeline running until the new one has survived a full week of production quirks unattended.

Expect the first month to be humbling. In our experience most early failures come from contracts nobody wrote down rather than from model quality. Fix the interfaces first and the models usually have enough headroom to surprise you on the upside.

Case study: BlockFrame Labs

We run our own publishing pipeline this way. One agent scans RSS feeds and ranks items against our topic list. A second checks candidates against recent posts to filter repeats. A third drafts. A fourth audits the draft against our house style rules before anything reaches a human editor.

None of these agents is impressive alone, and that's the point. The ranking agent fails cheaply and often; the orchestrator retries with backoff and moves on. Before the split, one long prompt did everything and produced inconsistent output we couldn't diagnose. After the split, every stage has a measurable pass rate, and when quality drops we know within one run which agent needs attention.

The economics changed too. Small fast models handle filtering and checking, and expensive frontier calls happen only where judgment matters. Cost per published post fell by roughly half while weekly throughput doubled, mostly because retriable stages stopped re-running the expensive ones.

Future outlook

Interoperability standards are arriving fast. Tool protocols like MCP and emerging agent-to-agent messaging standards are pushing toward a world where agents from different vendors join the same workflow. Assembly gets cheaper, governance gets harder, because your orchestrator will depend on components you don't control and can't fully audit.

Expect evaluation to become its own discipline inside engineering teams, separate from development. As pipelines grow, the bottleneck stops being capability and becomes verification: proving that a dozen cooperating agents did the right thing on an ordinary Tuesday afternoon. Teams building replayable traces and automated checks now will adopt each new model wave in days; teams without them will spend quarters.

Key takeaways

Multi-agent systems in production succeed on plumbing, not brilliance. Split responsibilities narrowly, put one orchestrator in charge, persist every piece of state outside conversations, and give each handoff a timeout and a fallback.

Start smaller than feels necessary. Two agents with clean contracts beat six agents in a tangled graph, and adding specialists later is easy once the first two prove themselves. When something misbehaves, fix the interface before blaming the model.

If you're evaluating vendors, ask one question before any demo: tell me about your last production failure. Anyone who has really run agents in production has a pile of these stories, and the good ones will happily explain what broke and what they changed.