Retry vs Circuit Breaker: Two Resilience Patterns for Agent Tool Calls

An agent that calls external tools will eventually call one that fails, and the two disciplined responses to that failure look similar enough on the surface to be confused, which is exactly why they get confused on the exam. The Retry pattern reattempts an operation that failed because of a brief, self-correcting problem — a network blip, a timeout, a moment of contention. The Circuit Breaker pattern does the opposite of retrying: it stops calling an operation altogether once failure looks less like a blip and more like a pattern, and it holds off until there is evidence the dependency has recovered. Both patterns exist to keep an agent alive through failure rather than crash on the first exception, but they are aimed at two different failure shapes, and applying the wrong one to a given shape either wastes effort or actively makes things worse.

That distinction — transient fault versus persistent fault, reattempt versus fail-fast — is the spine of this lesson, and it is worth taking seriously beyond exam mechanics. A tool-calling agent in production is, underneath the prompting and the reasoning loop, a distributed system making network calls to APIs, databases, and other services, and distributed systems fail in exactly these two shapes. Getting the response wrong is not a hypothetical: retrying a payment call without checking idempotency can double-charge a customer, and hammering an already-overloaded service with aggressive retries can be the difference between a brief outage and a cascading one. Domain 2 of the NCP-AAI exam blueprint calls this pairing "the most testable, most confusable" material in the whole domain, and this lesson exists to make the two patterns permanently distinguishable.

1. The Retry pattern: reattempting a transient fault

The Retry pattern transparently reattempts an operation that failed because of a transient fault — a fault expected to resolve itself given a little time or a fresh attempt. The canonical examples are a brief network blip, a temporary unavailability window, or a timeout that fired because a downstream service was momentarily slow rather than actually down. The defining property of a transient fault is that trying again, with no other change, has a real chance of succeeding, because nothing about the request itself was wrong — the problem was transient conditions in the network or the dependency, not a defect in what you asked for.

That last clause matters more than it looks. A validation error — a malformed request body, a missing required field, an operation the caller was never authorized to perform — is not transient. Retrying it changes nothing, because the input was wrong, not the timing. The Retry pattern is deliberately scoped to faults where the cause is time-bound. When the cause is not time-bound, retrying is wasted work at best and actively harmful at worst, which is the reasoning behind the first of the three retry strategies below.

The three retry strategies

The pattern names three concrete responses to a failed call, and choosing among them is itself a graded decision, not a single on/off switch:

Strategy	When it applies
Cancel	The fault is not transient — a validation error, a permissions failure, a malformed request. Retrying will not help, so stop and surface the error.
Retry immediately	An unusual, rare fault where an instant reattempt is likely to succeed — a momentary blip with no reason to believe the condition persists.
Retry after delay	The common case: busy or connectivity faults, where waiting gives the dependency time to recover before you try again.

Cancel is easy to underweight, because "retry" is the pattern's headline word and it is tempting to treat every failure as an invitation to retry. But cancel is a first-class branch of the pattern, not a fallback outside it — recognizing that a failure is not transient and should not be retried is as much a part of implementing Retry correctly as choosing a backoff schedule is.

Retry-after-delay is where most production retry logic actually lives, because most transient faults are busy/connectivity faults rather than rare instantaneous blips. The delay itself can grow in one of two shapes: incrementally, adding a fixed amount to the wait after each failed attempt, or exponentially (exponential backoff), doubling or otherwise multiplying the wait after each attempt. Exponential backoff is frequently paired with jitter — a small randomized offset added to the delay — specifically so that a fleet of clients that all failed at the same instant does not then all retry at the same instant again, which would recreate the exact spike in load that caused the failure in the first place.

Design considerations that go with any retry policy

A retry policy is not just "try again N times." A handful of considerations separate a retry implementation that helps from one that quietly makes things worse:

• Idempotency first. This is the precondition the next section is devoted to, and it comes before every other consideration: only retry operations that can run more than once without duplicating side effects.
• Tune the policy to the fault type. A rare instantaneous blip and a sustained busy period are different faults calling for different strategies (immediate versus delayed), and treating every failure identically wastes the information the failure mode itself carries.
• Avoid nested retry layers. If a tool call is wrapped in a retry policy, and that tool call itself sits inside a caller that also retries, the attempts multiply — three retries at one layer inside three retries at another layer is nine attempts, not six, and the multiplication compounds however many layers exist.
• Log failures. A retry that eventually succeeds can hide a developing problem if nobody is watching the failure count; logging keeps the pattern from becoming a silent tax on latency.

The most consequential of these is the warning the source material calls out explicitly as a trap: aggressive retries against a service that is already overloaded make the overload worse, not better. Every retried request is additional load on a dependency that is failing precisely because it has too much load already. For a fault that looks brief, immediate or short-delay retry is reasonable. For a fault that is lasting — the dependency has been failing for tens of seconds or minutes, not milliseconds — the correct response is to stop retrying and hand the problem to a circuit breaker instead. That handoff, from "keep trying" to "stop calling entirely," is the seam between the two patterns this lesson is about, and it is worth sitting with before moving to the second pattern: Retry's entire value proposition assumes the fault is short-lived, and the moment that assumption breaks, continuing to retry is no longer resilience — it is participation in the outage.

2. Idempotency: the precondition that makes retry safe at all

Idempotency is not a feature of the Retry pattern; it is the gate the Retry pattern must check before it is allowed to run at all. An operation is idempotent if running it more than once produces the same end state as running it exactly once — repetition does not compound the effect. A read (fetch this record) is naturally idempotent: reading twice returns the same data and changes nothing. Setting a value to an absolute state ("set the account's tier to gold") is idempotent, because doing it five times leaves the account in the same tier as doing it once. Incrementing a counter, sending an email, or — the example the source material uses directly — charging a payment are not idempotent by default: retry a naive "charge $50" call twice because the first response never arrived, and the customer may be charged twice, even though from the caller's point of view nothing went wrong except an unlucky timeout.

This is why "is the operation idempotent" is the first question in any retry decision, not an afterthought bolted on later. Before adding retry logic to a tool call, you must first establish that the operation is safe to run more than once — and if it is not naturally idempotent, you need a safeguard that makes it effectively idempotent before retry logic is allowed near it. The most common safeguard is an idempotency key: the caller generates a unique token for the logical operation once, attaches it to every attempt (including retries), and the receiving service uses that token to recognize "I have already applied this exact operation" and returns the original result instead of applying the effect a second time. Well-designed tools in an agent's toolkit should expose clear input/output contracts and be idempotent wherever the underlying operation allows it, precisely because that is what makes it safe for an agent framework to wrap those tools in a general-purpose retry policy without auditing every individual tool's side effects by hand.

Idempotency also explains why the Retry pattern's "cancel" branch exists for non-transient faults in a way that goes beyond "retrying won't help." Even if a validation failure would somehow succeed on a second attempt, retrying a non-idempotent operation without having verified idempotency is a risk you should not take merely because the first strategy in the table happens to be available. The rule generalizes past payments: any tool that mutates state — creates a record, sends a notification, decrements inventory, appends to a queue — is a candidate for double-application under retry unless something in the design (an idempotency key, a natural "set to absolute value" semantics, a dedupe check on the receiving side) prevents it. Treat idempotency as a property you verify, not one you assume, and the entire Retry pattern becomes safe to apply broadly; skip that check, and every retry policy you write is a policy for silently duplicating side effects under exactly the failure conditions — network flakiness — that retries are supposed to handle gracefully.

3. The Circuit Breaker pattern: a state machine, not a retry variant

Where Retry answers "how do I get past a brief hiccup," the Circuit Breaker pattern answers a different question: "how do I stop hammering a dependency that is not having a brief hiccup at all." The Circuit Breaker prevents an application from repeatedly calling an operation that is likely to keep failing. Its purpose is twofold: it gives the failing dependency room to recover instead of burying it under continued load, and it prevents cascading failure — the situation where one struggling dependency, kept under constant retry pressure by every caller that depends on it, drags the callers themselves into failure, and their callers in turn, until an isolated problem becomes a system-wide one.

The mechanism is a state machine with exactly three states, and the exam treats naming only two of them — typically Closed and Open, leaving out Half-Open — as one of the most common ways candidates get this wrong. All three states, and the behavior and transition that define each, are summarized here and then discussed individually below.

The three states and their transitions

State	Behavior	Transition out
Closed	Normal operation — requests flow through to the dependency, and failures are counted.	Failures exceed a threshold within a time window → moves to Open.
Open	Requests fail fast: the breaker does not even attempt the call, returning an error immediately, for a fixed timeout period.	After the timeout elapses → moves to Half-Open.
Half-Open	A limited number of trial requests are allowed through to test whether the dependency has recovered.	All trial requests succeed → back to Closed (failure counter resets). Any trial request fails → back to Open (timeout restarts).

Reading the table left to right tells the whole story: Closed is where the breaker starts and where it wants to return to, Open is a deliberate refusal to participate in an ongoing failure, and Half-Open is the controlled, low-risk way of finding out whether refusal is still warranted.

Closed — normal operation, with a failure counter running

In the Closed state, the breaker is invisible: every request the agent makes passes through to the real dependency exactly as if no breaker were present. What the breaker is doing underneath that transparency is counting failures within a rolling time window. As long as the failure rate stays under the configured threshold, nothing changes — a handful of isolated failures inside a busy service is normal and does not indicate the dependency is in trouble. It is only when failures accumulate past the threshold inside the window that the breaker concludes the dependency's problem is not an isolated blip, and trips.

Open — failing fast on purpose

Once tripped, the breaker moves to Open, and its behavior inverts completely: instead of forwarding requests, it rejects them immediately, without ever attempting the underlying call. This is fail fast, and it is worth naming explicitly because an Open breaker rejecting every request looks, from the outside, exactly like an outage — which is precisely the misreading the exam flags as a trap. A tripped breaker failing fast is a feature working as designed, not a bug: it is refusing to add more load to a dependency that has already demonstrated it cannot handle the current load, and it is sparing the calling application the cost (latency, thread occupancy, resource exhaustion) of waiting on calls that were very likely to fail anyway. The Open state persists for a fixed timeout period — long enough, by design, that the dependency has a real window to recover rather than being probed again within milliseconds.

Half-Open — the controlled recovery test

When the Open timeout expires, the breaker does not simply flip back to Closed and resume full traffic — that would risk slamming a still-fragile dependency with the exact volume of requests that caused the original failure. Instead it moves to Half-Open, where it allows through a limited number of trial requests specifically to test whether the dependency has recovered, while continuing to fail fast on everything else. If those trial requests succeed, the breaker concludes the dependency is healthy again, returns to Closed, and resets its failure counter to start counting fresh. If even one trial request fails, the breaker concludes the dependency is still unwell, goes straight back to Open, and the timeout period restarts. The Half-Open state exists specifically to avoid flooding a service that is still recovering — it is the pattern's answer to the same problem that jitter solves for Retry: don't let recovery itself become the next thing that causes a failure, by testing carefully instead of resuming at full volume.

4. Why Retry alone cannot do a circuit breaker's job

It is worth being explicit about why these two patterns are not two implementations of the same idea, because the exam's most common trap is exactly that conflation. Retry is stateless with respect to the dependency's overall health: each call is evaluated on its own, and the policy (cancel, retry immediately, retry after delay) is chosen based on what kind of fault this particular failure looks like. Retry has no memory of how many times the last ten calls to this dependency failed, and no notion of "this dependency's problem is not going away." That is not a missing feature so much as a design boundary — Retry is built to solve individual transient faults, and it solves that problem well.

The Circuit Breaker, by contrast, is entirely about memory and aggregate health: it tracks a failure count across a window, decides when that count crosses from "normal noise" into "this dependency is in trouble," and changes its own behavior — not the operation's behavior — as a result. A retry policy applied to a persistently failing dependency will, at best, waste latency retrying calls doomed to fail, and at worst, if the dependency is failing because it is overloaded, actively deepen that overload with every retried attempt, which is the aggressive-retry trap called out earlier. Retry has no off switch for this scenario built into itself; supplying that off switch, and knowing when to flip it, is the circuit breaker's entire job.

5. Retry vs Circuit Breaker: the comparison that resolves scenario questions

The table below is the centerpiece distinction for this domain — the single artifact that, read carefully, resolves most of the scenario questions the exam builds around this material.

	Retry	Circuit Breaker
Protects against	A transient fault — a brief, self-correcting problem in an individual call	A persistently failing dependency — a problem that is not going away on its own
Unit of decision	Each individual call, judged on its own	The dependency's aggregate health over a rolling window of calls
What happens on failure	The operation is reattempted (immediately, after a delay, or canceled if non-transient)	Once failures cross a threshold, the breaker trips Open and stops attempting calls entirely
Precondition to use safely	The operation must be idempotent, or protected by an idempotency safeguard	None on the calling operation itself — the breaker protects the caller, not the operation's side effects
State carried between calls	None — stateless per call	Yes — a failure counter, a current state (Closed/Open/Half-Open), and a timeout clock
Effect on load to the dependency	Can increase load (more attempts against the same dependency)	Reduces load (fails fast instead of calling at all, once Open)
When you'd reach for it alone	The fault is rare, brief, and the dependency is otherwise healthy	The fault has already lasted well past what a retry-and-move-on approach would tolerate
When you'd use both together	Retry through the breaker for short transient faults while it is Closed; stop retrying the instant the breaker signals Open	Same scenario — the two are complementary layers, not alternatives

That last row is the point worth internalizing above the rest: Retry and Circuit Breaker are not competing answers to the same multiple-choice question in real production systems — they compose. A well-built resilience layer retries through the breaker: while the breaker is Closed, individual failed calls still get the retry treatment appropriate to their fault type, and the breaker is meanwhile counting those failures in the background. If the count crosses the threshold, the breaker trips, and at that point the correct behavior is to stop retrying — the breaker itself becomes the reason further calls fail fast, which is a cheaper and safer failure than a retry loop discovering the same thing the slow way, call after call. Mature resilience libraries build exactly this composition in as a first-class feature rather than leaving it to be assembled by hand: Polly in the .NET ecosystem and Resilience4j in the Java ecosystem both ship Retry and Circuit Breaker as separate, stackable policies for precisely this reason.

6. Worked example: a recommendation service that starts failing

Consider an agent that, as part of answering a user's question, calls a downstream recommendation service to fetch personalized suggestions. Trace what happens under three different resilience configurations as that service degrades.

No resilience pattern at all. The recommendation service returns a connection timeout on the very first call. The exception propagates up uncaught, and the agent's turn ends in an error the user sees directly — a single blip anywhere in this dependency chain takes down the whole interaction, regardless of whether the blip would have cleared itself half a second later.

Retry only, no breaker. The agent wraps the call in a retry policy: on failure, wait with exponential backoff and try again, up to a handful of attempts. The first few times the recommendation service hiccups briefly, this works exactly as intended — the retry succeeds on the second or third attempt, and the user never notices anything happened. Now suppose the recommendation service's database has actually gone down, not for half a second but for several minutes. Every call the agent makes now exhausts its full retry budget before finally giving up, so every user request that touches recommendations pays the full cost of several retries' worth of backoff delay before failing anyway — and if enough concurrent users are hitting the agent during this window, the aggregate retry traffic lands on the already-struggling recommendation service as additional load precisely while it is trying to recover, which is the aggressive-retry trap in action. Retry alone has converted a dependency outage into a slow, resource-consuming outage for every caller, and it has made the underlying outage marginally worse in the process.

Retry plus Circuit Breaker. The same retry policy is now wrapped inside a circuit breaker, Closed by default. During the brief hiccup case, nothing changes — the failures are isolated, well under threshold, and retry resolves them exactly as before, invisibly. When the sustained multi-minute outage begins, the first handful of calls fail (each retried the same way as before), but because those failures land inside the counting window, the breaker's threshold is crossed quickly and it trips to Open. From that point until the timeout expires, every call to the recommendation service fails immediately — no network attempt, no retry delay, no wasted latency — and the agent's calling code can catch that fast failure and fall back gracefully (skip personalization for this turn, or serve a cached or generic recommendation) instead of making the user wait through a retry budget that was never going to succeed. Meanwhile the recommendation service, no longer receiving the extra load from every caller's retries, gets the room to actually recover. When the Open timeout elapses, the breaker moves to Half-Open and sends a small number of trial calls. If the service is back, those succeed, the breaker closes, and normal retry-covered traffic resumes; if the outage is still ongoing, the trial fails, the breaker reopens, and the cycle repeats without ever exposing the full weight of traffic to a service not ready for it.

The comparison across these three configurations is the whole lesson in miniature: no pattern means any failure is catastrophic; Retry alone handles brief faults well and handles sustained faults badly, including making them worse; Retry plus Circuit Breaker handles both, because each pattern is doing the part of the job the other was never designed to do.

7. Why this pairing is on the NCP-AAI exam

Domain 2, Agent Development, carries 15% of the NCP-AAI blueprint — tied with the Architecture domain for the heaviest weight of any single domain in the exam. Within Domain 2, error handling with retry logic and graceful failure recovery is its own numbered objective (2.4), sitting directly alongside the objective on building and connecting custom tools (2.3) that gives retry logic something to protect in the first place: well-designed tools expose clear input/output contracts and are, where possible, idempotent, and that idempotency is exactly what makes retrying those tools safe. The domain's own framing calls the Retry and Circuit Breaker state machines "the most testable, most confusable" material it contains, which is a direct signal about where scenario questions concentrate: not on defining either pattern in isolation, but on presenting a failure description and asking which pattern — or which state, or which strategy — fits it.

Expect the question shape to work like this: a short scenario describes a dependency's failure behavior (a single timeout versus minutes of sustained failures, an idempotent read versus a non-idempotent write, a service that has just come back online after an outage), and four answer options offer different resilience responses, exactly one of which matches the failure shape described. Recognizing "this has been failing for minutes" as a cue for the breaker rather than more aggressive retrying, recognizing "before you retry this, check whether it's idempotent" as a precondition rather than a nice-to-have, and recognizing Half-Open as the state that exists specifically to test recovery without flooding the dependency, are the three recurring load-bearing facts behind most of this domain's error-handling items. A candidate who has the vocabulary but conflates the two patterns, or who forgets the third state exists, will read as though they understand resilience while actually missing the mechanism the exam is testing.

8. Common mistakes with Retry and Circuit Breaker

Mistake	What actually goes wrong	Fix
Retrying a non-idempotent operation with no safeguard	A payment, an email send, or a record creation gets applied twice because the first attempt's response never arrived, even though the operation itself "succeeded"	Verify idempotency, or add an idempotency key, before any retry policy is allowed to touch the operation
Treating every failure as retry-worthy	A validation or permissions error gets retried repeatedly and fails identically every time, wasting latency and attempts	Classify the fault first — non-transient faults belong in the cancel branch, not the retry loop
Nesting retry layers	A retry policy at the tool layer sits inside another retry policy at the caller layer, multiplying total attempts (three retries inside three retries is nine, not six)	Put retry logic at one layer only, and make that ownership explicit
Retrying aggressively against an overloaded dependency	Every retried call adds more load to a service that is already failing because of too much load, deepening and prolonging the outage	For sustained faults, stop retrying and hand the decision to a circuit breaker instead
Implementing only Closed and Open, skipping Half-Open	The breaker never has a controlled way back to service — it either stays permanently tripped past when it should reopen, or someone manually flips it back to full traffic and re-triggers the original failure	Implement all three states; Half-Open's limited trial requests are what make recovery safe
Reading an Open breaker as an outage bug	An on-call engineer sees every call to a dependency failing instantly and assumes something is broken in the breaker itself	Recognize fail-fast as intentional: the breaker is refusing to add load to a dependency it has already determined is unhealthy
Treating Retry and Circuit Breaker as alternatives	A system implements one or the other and is under-protected against the failure shape the missing pattern was built for	Use them together: retry through a Closed breaker for transient faults, and let the breaker's Open state stop retries once a fault proves persistent

9. Glossary recap

Term	One-line definition
Transient fault	A brief, self-correcting failure — a network blip, temporary unavailability, or timeout — where a fresh attempt has a real chance of succeeding
Retry pattern	Reattempting a failed operation, using one of three strategies: cancel, retry immediately, or retry after delay
Exponential backoff	A retry delay schedule that grows multiplicatively after each failed attempt
Jitter	A small randomized offset added to a retry delay so that many clients failing together do not all retry together
Idempotent operation	An operation that produces the same end state whether it runs once or many times, making it safe to retry
Idempotency key	A unique token attached to every attempt of a logical operation so a receiving service can recognize and ignore duplicate attempts
Circuit breaker	A state machine (Closed → Open → Half-Open) that stops calling a dependency it has determined is persistently failing
Closed (state)	Normal operation — requests flow through, and failures are counted against a threshold
Open (state)	The breaker fails fast, rejecting requests without attempting the call, for a fixed timeout period
Half-Open (state)	A limited number of trial requests test whether the dependency has recovered, before resuming full traffic or reopening
Fail fast	Rejecting a request immediately rather than attempting a call likely to fail — an Open breaker's intended behavior, not a malfunction
Cascading failure	One struggling dependency dragging its callers, and their callers in turn, into failure under sustained retry pressure

10. Key takeaways

• Retry and Circuit Breaker solve two different failure shapes: Retry answers a brief, self-correcting fault; the breaker answers a dependency that is persistently failing.
• Before adding retry logic to any operation, confirm it is idempotent — or protected by an idempotency safeguard — because retrying a non-idempotent operation without one can duplicate side effects.
• The three retry strategies are cancel (non-transient fault), retry immediately (rare blip), and retry after delay, typically with exponential backoff and jitter (common busy/connectivity faults).
• The circuit breaker is a three-state machine: Closed (normal, failures counted) → Open (fail fast, on purpose, for a timeout) → Half-Open (limited trial requests test recovery) → back to Closed or Open depending on the trial's outcome.
• Half-Open exists specifically so recovery testing does not itself flood a still-fragile dependency.
• Aggressive retries against an already-overloaded dependency make the outage worse; for a sustained fault, stop retrying and let the breaker take over.
• The patterns compose rather than compete: retry through a Closed breaker for transient faults, and let the breaker's Open state stop retries once a fault proves persistent.
• On the exam, expect scenario questions that describe a failure's duration and shape and ask which pattern, strategy, or state fits — the fix for nearly all of them is naming the fault as transient or persistent first.

11. Next: building and connecting custom tools with clear input/output contracts

Everything in this lesson assumed there was a tool call worth protecting — a well-defined boundary between the agent and an external system, with inputs going out and an observation coming back. What that boundary needs to look like before resilience logic can even be layered onto it is its own objective in this domain, and it is the natural next step: designing tools with clear input/output contracts, and building in the idempotency this lesson leaned on repeatedly, so that whatever wraps those tools — a retry policy, a circuit breaker, or both — has something safe to wrap.

Next: Building and connecting custom tools with clear input/output contracts — how the NeMo Agent Toolkit's "build once, reuse" model of agents, tools, and workflows as composable function calls shapes a tool contract that a resilience layer, and not just a single caller, can depend on.