Multi-Agent Orchestration Topologies: Centralized, Decentralized, Federated, Hierarchical

A multi-agent system is not defined by how many language models are running; it is defined by how those models are wired together. Two systems can each run six specialist agents and behave nothing alike in production, because one routes every decision through a single supervisor and the other lets its agents negotiate directly with no controller at all. The wiring — the orchestration topology — is what decides who can fail without taking the rest down, how much latency a decision costs, and who is accountable when two agents disagree.

That is why topology gets its own vocabulary on the exam rather than being folded into "multi-agent systems" as a single idea. Centralized, decentralized, federated, and hierarchical control structures solve the same underlying problem — getting several specialized agents to cooperate on one task — but they solve it with different tradeoffs, and a scenario question is almost always asking you to match a described set of constraints to the topology built to handle them.

1. What a multi-agent system actually is

A multi-agent system (MAS) is a team of specialized agents that collaborate on a complex problem. Each agent in that team typically has a defined role, some degree of autonomy in how it pursues that role, and — this is the detail that trips people up — only a local view of the system. No single agent in a well-designed MAS necessarily sees the whole picture. A research agent knows how to search and summarize; it does not necessarily know what the writing agent downstream will do with its output, or whether a third agent is simultaneously fact-checking the same claim.

That local-view property is the cleanest test for whether something is genuinely multi-agent. A system where one model calls a search tool, then a calculator tool, then a code-execution tool is still a single agent — one reasoning process, one full view of the task, tools that have no role or autonomy of their own. Multiplying the number of tools does not multiply the number of agents. A multi-agent system requires multiple agents: separate reasoning loops, each with a role, each making its own local decisions, each potentially built on a different model or prompt tuned for its specialty.

This distinction matters because "agent" and "tool" get blurred constantly in casual descriptions, and the exam's own trap list calls it out directly: a single agent with many tools is not automatically a multi-agent system. If a described system has one planner deciding every step and simply invoking capabilities, it is an agent with a toolbelt. If it has multiple planners, each capable of independent reasoning about its slice of the problem, it is a MAS — and only once you have a MAS does the question of orchestration topology even apply.

Why teams of agents instead of one bigger agent

The motivation for splitting into a team rather than building one more capable agent is threefold. Specialization lets each agent be tuned — through prompting, tools, or fine-tuning — for a narrow job it does well, rather than asking one generalist prompt to be simultaneously good at retrieval, arithmetic, and tone. Modularity and scalability mean you can replace or upgrade one agent without retraining or re-prompting the whole system; swap the pricing agent for a better one and the rest of the team is untouched. And MAS designs carry built-in transparency in a way a monolithic agent does not: because each agent's role is scoped, you can trace which agent produced which decision, which is a meaningfully easier debugging and audit story than untangling one enormous chain-of-thought.

None of those benefits are free, and the cost is exactly what orchestration topology exists to manage.

2. Why the control structure is the variable that matters

Once you accept that a MAS is a team of role-bearing, semi-autonomous agents, the next question is: who decides what happens next? That is the entire content of "orchestration topology" — it is a description of the control structure layered over the team, not a description of the agents themselves.

Two MAS deployments with identical agent rosters — say, a retrieval agent, a drafting agent, and a compliance-review agent — can have opposite operational profiles depending on topology alone. Route every handoff through a supervisor, and you get one throat to choke when output is wrong, at the cost of that supervisor becoming a bottleneck and a single point of failure. Let the three agents negotiate directly with no controller, and you eliminate that bottleneck, but you also eliminate the one place that used to catch conflicting actions before they shipped. The agents did not change. The topology did, and the failure mode moved with it.

This is why NVIDIA's glossary treats topology as a first-class concept with its own named categories rather than leaving orchestration as an implementation detail. Four topologies cover the space the exam expects you to recognize: centralized, decentralized, federated, and hierarchical.

3. Centralized orchestration: one supervisor directs the team

In a centralized topology, one supervisor agent (or a fixed orchestration layer) directs the team. Sub-agents receive tasks from that supervisor, execute them, and report results back up; they do not typically communicate laterally with each other or make independent decisions about the overall plan. The supervisor holds the global view that no individual specialist has, decomposes the task, assigns pieces to the right specialist, and reconciles the results into a final answer.

The advantage is exactly what you would expect from putting one entity in charge: clear accountability and simpler coordination. When something goes wrong, there is one place to look — the supervisor's decision log — rather than reconstructing a negotiation among peers. Conflicting sub-agent outputs get resolved by the supervisor rather than by an ad hoc protocol between the sub-agents themselves, which keeps the coordination logic in one place instead of scattered across every pairwise agent relationship.

The cost is equally direct. The supervisor is a single point of failure and a potential bottleneck: if it goes down, misroutes work, or simply cannot keep up with throughput, the whole system stalls regardless of how healthy the specialist agents are. Centralized topologies also tend to concentrate context-window and latency pressure on one node, since every result funnels through the supervisor before anything downstream can happen. This is the topology most people picture by default when they hear "multi-agent system," which is precisely why the exam's trap list warns that multi-agent does not imply centralized supervision — it is one option among four, not a synonym for the category.

4. Decentralized orchestration: autonomous peers, no central controller

A decentralized topology removes the supervisor entirely. Autonomous peer agents share information directly with one another — through shared state, message-passing, or a common blackboard — and no single agent holds authority over the others' decisions. Each peer reasons locally and acts on what it can see and what its peers tell it.

The payoff is resilience and the absence of a single bottleneck. Because no one node is load-bearing for the whole system's coordination, the loss or degradation of any one peer does not automatically halt the others; the remaining agents can often continue operating on the information they already have. This makes decentralized topologies attractive when uptime under partial failure matters more than perfectly consistent global behavior — swarms of agents monitoring different data streams, for instance, where any individual monitor going offline should not stop the others from reporting.

The tradeoff is coordination cost pushed down into the agents themselves. Without a supervisor to arbitrate, conflicting actions — two peers both deciding to act on the same resource, or reaching contradictory conclusions — have to be resolved by whatever protocol the peers share, and if that protocol is weak, the system can genuinely thrash. This is exactly why the exam's trap list singles out the misconception that decentralized means leaderless chaos: decentralized is not the absence of governance, it is governance implemented as a shared protocol among equals rather than as a hierarchy. A decentralized system with a well-designed consensus or shared-state mechanism is orderly; a decentralized system with no coordination mechanism at all is the failure mode people wrongly assume is the whole category.

5. Federated orchestration: cooperation across organizational boundaries

A federated topology is built for a specific circumstance the other three do not address well: agents that need to cooperate but belong to different organizations, systems, or trust domains. Federated agents cooperate across organizational boundaries via shared protocols rather than shared infrastructure or a shared controller. No single company's supervisor can sit above another company's agents, so the coordination mechanism has to be a negotiated interface both sides agree to honor — a defined message format, an API contract, a set of permissions — rather than direct control.

The best-fit scenario is cross-company or cross-domain collaboration: a logistics agent at one firm coordinating with a customs-clearance agent at a government system, or a hospital's scheduling agent negotiating with an insurer's authorization agent. Each side keeps its own internal topology private — a hospital might run its own agents in a centralized arrangement internally — and federation only governs the seam between organizations.

The distinguishing cost of federation is protocol and trust overhead. Because no party can simply inspect or override the other's internals, the shared protocol has to anticipate disagreement, latency, and partial information at the boundary, and versioning that protocol as either side's internal agents evolve is a coordination problem that centralized and decentralized topologies never have to solve, because in those cases everything sits inside one administrative domain.

6. Hierarchical orchestration: tiered supervision for scale

A hierarchical topology organizes agents into tiered layers of supervision rather than the flat one-supervisor-over-many-workers shape of a pure centralized system. A top-level orchestrator delegates to mid-level supervisors, each of which in turn manages its own team of specialist agents, and results roll back up the same tiers they came down through. Picture a regional-manager layer between a single CEO agent and dozens of frontline specialist agents: no one node has to hold both the global plan and every low-level execution detail at once.

The best fit is large systems needing structured delegation — enough agents that a single supervisor cannot reasonably coordinate all of them directly without becoming exactly the bottleneck a centralized topology risks. Hierarchy solves that by bounding each supervisor's span of control: the top orchestrator only has to manage a handful of mid-level supervisors, and each mid-level supervisor only has to manage its own smaller team.

Hierarchy is easy to mistake for a more elaborate version of centralized, and the exam expects you to keep them distinct. Centralized has one controller for the whole team, however large; hierarchical distributes that control across tiers precisely because the team has grown too large for a single controller to remain the clear-accountability advantage it is at smaller scale. The tradeoff hierarchy accepts in exchange for scale is added latency — a decision may need to travel up two tiers and back down before it is finalized — and a subtler failure mode: a failure at a mid-level supervisor now has a blast radius of everything beneath it in that branch, which is bigger than any single leaf agent's blast radius but smaller than the whole system's, unlike a single centralized supervisor whose failure takes down everything at once.

7. The four topologies side by side

The comparison is the fastest way to hold all four in your head at once, and it is the shape most scenario questions are testing against directly.

Topology	Control structure	Strength	Primary risk	Best-fit scenario
Centralized	One supervisor directs the team	Clear accountability, simpler coordination	Supervisor is a single point of failure / bottleneck	Small-to-medium team, one clear owner of the final answer
Decentralized	Autonomous peers share information directly, no central controller	Resilience, no single bottleneck	Conflicting actions with no arbiter unless the shared protocol is strong	Swarm-style monitoring, high availability under partial failure
Federated	Agents cooperate across organizational boundaries via shared protocols	Cross-company / cross-domain collaboration without ceding control	Protocol and trust overhead at the boundary; versioning across domains	Multi-organization workflows (supply chain, healthcare, cross-vendor integrations)
Hierarchical	Tiered layers of supervision	Structured delegation at scale	Added latency across tiers; a mid-tier failure takes its whole branch down	Large systems where one supervisor cannot cover the whole team directly

Notice what does not appear anywhere in that table: agent count as the deciding column. A twenty-agent system and a four-agent system can both legitimately be centralized, and a four-agent system can legitimately be decentralized. The row you pick is a function of accountability needs, failure tolerance, organizational boundaries, and scale — not headcount.

8. Agent-to-agent communication and coordination

Whichever topology you choose, agents inside it have to actually exchange information, and that exchange is governed by agent-to-agent communication protocols. These protocols cover work decomposition (how a task gets split into pieces each agent can own), resource distribution (which agent gets which tool, budget, or dataset access), conflict resolution (what happens when two agents produce contradictory outputs), and cooperative planning (how agents align on a shared sequence of steps). Agents can converse in natural language directly, or coordinate indirectly through shared state — a blackboard, a shared memory store, or a task queue that every agent reads from and writes to.

Coordination protocol is where topology and communication meet. A centralized topology can afford a comparatively thin protocol, because the supervisor absorbs most of the coordination logic itself — sub-agents mostly just need to report results in a format the supervisor understands. A decentralized topology needs a much richer protocol, because the coordination logic that used to live in a supervisor now has to be encoded into how peers negotiate, and a weak protocol is exactly what turns decentralized resilience into decentralized chaos. Federated topologies need the protocol to also carry authentication and authorization across the organizational seam, since a shared protocol without a trust boundary is not actually federation, it is just decentralization that happens to span two companies. Hierarchical topologies need the protocol to be tier-aware — a message format that distinguishes "this goes to my direct supervisor" from "this goes to a peer at my own tier" — or the tiers collapse into an accidental flat mesh.

The exam's own framing of MAS challenges tracks this closely: coordination and conflicting actions are the headline risk, and the mitigation named is an orchestration layer plus shared state — which is, in different proportions depending on topology, exactly what the communication protocol is providing.

9. Worked example: choosing a topology as a support system grows

Consider a customer-support system that starts small: one triage agent that reads an incoming ticket, decides which specialist to route it to, and hands off to either a billing agent or a technical-troubleshooting agent, then relays the specialist's answer back to the customer. This is naturally centralized — the triage agent is the supervisor, the two specialists never need to talk to each other, and accountability is trivial: every routing decision and every final answer passes through one node you can log and audit.

Now the company grows. Support now spans five departments — billing, technical support, shipping, returns, and a new AI-features desk — and ticket volume means the single triage agent is now a genuine bottleneck, queuing tickets during peak hours and becoming the one thing that takes the whole support pipeline down if it misbehaves. This is the signal to move toward hierarchical: introduce a mid-tier supervisor per department cluster (say, one supervisor over billing-and-returns, since refund logic touches both, and one over technical-and-features), with a thin top-level orchestrator that only has to route an incoming ticket to the right mid-tier supervisor rather than to one of a dozen individual specialists directly. The top orchestrator's span of control shrinks back to something manageable, and each mid-tier supervisor's span is similarly bounded.

Suppose next that shipping tickets increasingly require live cooperation with a third-party logistics partner's own automated system — checking a package's real-time location requires the partner's agent, not just your own. That single connection does not require redesigning your whole internal topology; it requires a federated seam at exactly that boundary. Your shipping supervisor's tier keeps its internal hierarchical structure, and it additionally speaks a shared protocol — a defined request/response contract — to the logistics partner's agent, which the partner operates and governs entirely on their own side.

Finally, imagine the company also deploys a fleet of proactive monitoring agents, each watching a different data stream (order-error rates, chat-sentiment trends, refund-velocity spikes) for anomalies worth escalating. These monitors do not need a supervisor telling them what to watch, and you specifically want the loss of any one monitor to not take the others down — that is a decentralized cluster sitting alongside the hierarchical support pipeline, each monitor sharing anomaly signals with its peers and independently deciding when something crosses a threshold worth escalating into the main ticket flow.

The lesson in that progression is that real systems rarely commit to exactly one topology forever. They start with whichever topology matches their current scale and boundary constraints, and they graft on a different topology at a specific seam when a new constraint — scale, an organizational boundary, or a resilience requirement — shows up that the original topology does not handle well.

10. Why this is on the NCP-AAI exam

Agent Architecture and Design is Domain 1 of the NCP-AAI blueprint, weighted at 15%, tied with Agent Development for the single largest domain on the exam. Orchestration topology sits inside that domain specifically because the exam's own scope note says candidates are expected to reason about why an orchestration topology fits a scenario, not merely recite the four names — the objectives explicitly use the verb "orchestrate" for multi-agent workflows, not "list" or "define."

Expect the concept tested in three recurring shapes. First, direct identification items: a short description of a control structure ("autonomous peers share information directly with no single supervisor") mapped to its name — decentralized, in that example — which is a pure vocabulary check but still costs points if the four names are not memorized precisely. Second, scenario-fit items: a described organizational or scale constraint (multiple companies must cooperate; a team has grown too large for one supervisor; uptime under partial failure matters more than perfect consistency) mapped to the topology built for that constraint. Third, trap items built directly from the misconceptions the source material flags: treating a single agent with many tools as a MAS, assuming decentralized means no governance at all, or assuming more agents automatically means better coordination when the real bottleneck is the orchestration layer, not the agent count.

Because Domain 1 is foundational, a shaky grasp of topology also weakens later domains. Evaluation and safety questions elsewhere in the blueprint frequently assume you already know which topology a described system uses, because the correct mitigation for coordination failures or governance gaps depends on it — a governance fix that works for a centralized supervisor is not the fix you would propose for a decentralized peer mesh.

11. Common mistakes with multi-agent orchestration topologies

Mistake	What it gets wrong	Correct framing
Treating a single agent with many tools as "multi-agent"	Confuses tool-calling breadth with agent count	MAS requires multiple agents with their own roles and some autonomy, not one planner with a large toolbelt
Assuming "multi-agent" implies centralized supervision	Collapses four distinct topologies into the one people picture by default	Decentralized, federated, and hierarchical are equally valid topologies with different failure and governance profiles
Assuming decentralized means leaderless chaos	Mistakes the absence of a controller for the absence of any coordination	Decentralized systems still need a shared protocol for conflict resolution; without one, they do become chaotic — the protocol is what prevents that, not luck
Believing more agents automatically improves coordination	Adds headcount without fixing the actual coordination mechanism	Coordination and conflicting actions are best solved with an orchestration layer and shared state, not by adding more agents
Picking a topology by agent count alone	Ignores accountability, organizational boundary, and resilience requirements	Topology choice is driven by who needs to own accountability, whether trust boundaries exist, and how much resilience to partial failure is required
Confusing hierarchical with "a bigger centralized system"	Misses that hierarchy exists specifically to bound each supervisor's span of control	Centralized has one controller regardless of scale; hierarchical distributes control across tiers once scale makes one controller a bottleneck
Assuming federation is just decentralization that spans two companies	Drops the trust-boundary and protocol-versioning requirement that federation specifically exists to handle	Federation requires a negotiated shared protocol at the organizational seam, including authentication and authorization, not just shared state

12. Glossary recap

Term	One-line definition
Multi-agent system (MAS)	A team of specialized, semi-autonomous agents collaborating on a task, each typically with a role and a local view
Local view	The property that no individual agent in a MAS necessarily sees the whole system's state or plan
Orchestration topology	The control structure governing how agents in a MAS coordinate — centralized, decentralized, federated, or hierarchical
Centralized topology	One supervisor directs the team; clear accountability, simpler coordination, single point of failure
Decentralized topology	Autonomous peers share information directly with no central controller; resilient, no single bottleneck
Federated topology	Agents cooperate across organizational boundaries via shared protocols rather than shared control
Hierarchical topology	Tiered layers of supervision that bound each supervisor's span of control at scale
Agent-to-agent protocol	The communication mechanism for work decomposition, resource distribution, conflict resolution, and cooperative planning
Shared state	A blackboard, memory store, or task queue agents read from and write to as an alternative to direct messaging
Coordination / conflicting actions	The primary MAS risk, mitigated by an orchestration layer plus shared state rather than by adding more agents
Unbounded recursive delegation	The failure mode where a sub-agent that can itself spawn further sub-agents loops or over-delegates without a depth limit

13. Key takeaways

• A multi-agent system is defined by multiple agents with roles and local views, not by tool count — a single agent calling many tools is still one agent.
• NVIDIA's glossary names four orchestration topologies: centralized (one supervisor, clear accountability, single point of failure), decentralized (autonomous peers, resilient, no bottleneck, needs a strong shared protocol), federated (cross-organizational cooperation via shared protocols), and hierarchical (tiered supervision for scale, at the cost of added latency and per-branch blast radius).
• Topology choice is driven by accountability needs, organizational trust boundaries, resilience requirements, and scale — never by agent headcount alone.
• Decentralized does not mean leaderless chaos; it means governance implemented as a shared protocol among equals instead of a hierarchy, and a weak protocol is what produces the chaos people wrongly attribute to the topology itself.
• Coordination and conflicting actions are the standing MAS risk across every topology, mitigated by an orchestration layer and shared state rather than by adding more agents.
• Real systems often mix topologies at different seams — a hierarchical core with a federated boundary to a partner organization and a decentralized monitoring cluster alongside it — rather than committing to exactly one topology everywhere.
• A specialist agent that can spawn further sub-agents needs an explicit delegation-depth limit; unbounded recursive delegation is the exam's flagged critical risk for this pattern, independent of which topology the system otherwise uses.

14. Next: resilience patterns for agent tool calls

Choosing the right topology only gets a multi-agent system coordinated on paper; it says nothing about what happens when an individual tool call inside that system starts failing under load. You do not yet know how an agent should behave when a downstream service is slow, flaky, or fully down — whether it should keep retrying, back off, or stop calling that service altogether for a cooldown period.

Next: Retry vs circuit-breaker resilience patterns for agent tool calls — the two standard responses to a failing dependency, why retrying blindly can turn a brief outage into a cascading failure across every agent that depends on the same tool, and when a circuit breaker's fail-fast behavior is the safer default for an orchestrated multi-agent system.