The Retry Pattern: Transient Faults, Backoff, and Idempotency

A failed call carries a reason, even when that reason is buried inside a generic exception or timeout, and the Retry pattern's first move is always to ask what produced the failure before asking what to do about it. Faults split into two families that behave completely differently under a second attempt.

A transient fault is one where the underlying cause is bound to a narrow window of time — a packet dropped in transit, a load balancer briefly routing to an instance that had not finished starting up, a downstream service hitting a momentary spike in concurrent requests. [GROUND TRUTH] (Sources/ncp-aai/domain-2-agent-development.md): a transient fault is defined directly as "a brief network blip, temporary unavailability, or a timeout." Nothing about the request itself was wrong; the environment the request traveled through was, for a moment, not cooperating. Try the exact same request again a moment later, and there is a real, non-zero chance the window has already closed.

A fault that is not transient has a cause rooted in the request's content, not in timing: a missing required field, a token that has expired, a permission the caller was never granted, a request body that fails schema validation. None of these resolve themselves by waiting. Sending byte-for-byte the same malformed request a second time produces byte-for-byte the same rejection a second time, because nothing about the world changed between the two attempts — only the clock did.

Fault categories a retry decision has to tell apart

The middle row is worth pausing on, because it is easy to misclassify. A 429 Too Many Requests response is not the same shape as a plain network blip — it usually comes with a specific signal (a Retry-After header, a stated cooldown) about exactly when retrying becomes worth doing, rather than an unknown, best-guess delay. Treating a rate-limit response as an ordinary transient fault and retrying on a generic schedule ignores information the failure itself handed you.

Once a fault is classified, the Retry pattern offers exactly three responses, and picking among them is the pattern's whole decision procedure — not a menu to combine loosely, but a branch to walk through in order.

First branch: is the fault transient at all? If not — the request was malformed, unauthorized, or asking for something that does not exist — the correct response is to cancel: stop, do not retry, and surface the failure so the caller (a human, or an earlier stage in a chain) can address the actual problem. Retrying a non-transient fault is not a neutral, harmless waste of one extra call; it delays surfacing a problem that needed a different fix (correcting the request, refreshing a credential) than "wait and try again," and every retried attempt against a non-transient fault is time the real fix is not getting applied.

Second branch: is this a rare, one-off blip? If the fault looks transient and there is no particular reason to think the underlying condition will still be present a moment from now — a single dropped packet is the textbook case — retry immediately, with no deliberate delay. The reasoning is that an instant reattempt has a real chance of finding conditions already back to normal, and adding an artificial wait before that reattempt would only add latency without adding any real benefit.

Third branch: is this a busy or connectivity fault that is likely to persist for a bit? This is the common case in practice, and it calls for retrying after a delay rather than immediately — waiting gives whatever caused the failure (contention, a brief outage, a saturated connection pool) actual time to clear before the next attempt adds to the same conditions that caused the first failure. This is also where the pattern's most consequential trap lives: retrying immediately, or retrying too aggressively, against a fault that is a busy-dependency problem adds load to a dependency that is failing because it already has too much load, which can turn a fault that would have cleared on its own into one that gets worse specifically because retries kept piling on top of it.

Retry-after-delay needs an actual delay value, and the schedule that delay follows is called backoff. [GROUND TRUTH] (Sources/ncp-aai/domain-2-agent-development.md): "Delays may grow incrementally or exponentially (exponential backoff, often with jitter)." Two shapes are in common use, and they are not interchangeable.

⚠️ UNVERIFIED: the source material does not state a universal numeric threshold for how many total attempts a production retry policy should allow before deferring to a longer-horizon mechanism — a specific cap (three attempts, five attempts) is an implementation choice each system has to set for itself, not a fixed exam fact, and should not be quoted as one.

L1 — Intuition

Picture ten callers who all hit the same brief outage at the same instant. If every one of them waits the same fixed amount of time and retries at the same instant again, they recreate the exact same simultaneous spike that likely contributed to the failure in the first place — the dependency sees ten requests, fails, sees ten requests again a fixed interval later, and so on. A backoff schedule that grows, combined with a small random offset per caller, spreads those ten retries out across a widening window instead of a single recurring instant, giving the dependency room to actually clear the backlog between waves.

L2 — Mechanism

Incremental backoff adds a fixed amount to the wait after each failed attempt: 1 second, then 2 seconds, then 3, then 4 — a linear climb. Exponential backoff multiplies the wait after each attempt instead of adding to it: 1 second, then 2, then 4, then 8 — a schedule that grows far faster after only a handful of attempts. Exponential is the more common default for connectivity and busy-dependency faults specifically because the failure conditions those faults describe (an overloaded dependency, a saturated network path) tend to need meaningfully more recovery time the longer they have already persisted, and a schedule that only adds a fixed increment each time under-responds to that reality, staying aggressive for far longer than a multiplicative schedule would.

Jitter is a small randomized offset added on top of whichever schedule is chosen — instead of waiting exactly 4 seconds, a caller waits 4 seconds plus or minus a random fraction of a second. Jitter's entire purpose is defeating the synchronized-retry problem the intuition above described: without it, every caller that failed at the same moment, on the same fixed schedule, retries at the same moment again, every single cycle, regenerating the load spike each time; with a small random offset attached, that same population of callers spreads its retries across a window instead of a point, and the dependency sees a smoother ramp rather than a series of synchronized waves.

L3 — The exam-relevant edge case: backoff without a retry cap is not a complete policy

A backoff schedule answers "how long to wait between attempts," but it says nothing about "how many attempts total," and a policy that specifies only the delay shape without also bounding the number of attempts can, for a fault that genuinely does not clear, keep an agent retrying indefinitely against a dependency that was never coming back on its own — precisely the situation this module's already-authored M2-05 addresses by handing that decision off to a different mechanism once a fault has clearly stopped looking transient. A Retry policy without an attempt cap is not self-limiting; it needs either a fixed maximum number of attempts, or a signal from outside the pattern itself (which is exactly what the sibling pattern in M2-05 supplies) telling it when to stop.

Everything above answers "should I retry, and how soon" — it says nothing about whether a second attempt is safe, and that is a separate question with its own answer, worked out in full in this module's M2-03: an operation is idempotent if running it more than once produces the same end state as running it once, and only idempotent operations, or non-idempotent ones protected by an idempotency-key safeguard, are candidates for any of the three responses above that involve trying again.

This is not a footnote to the pattern — it is a gate the pattern has to pass through before any backoff schedule or immediate-retry decision is allowed to matter. A read is safe to retry on any schedule, because a second read changes nothing. A payment charge, an inventory decrement, or a notification send is not safe to retry as-is, no matter how correctly the fault was classified as transient and no matter how well-tuned the backoff schedule is, unless it carries the idempotency-key safeguard M2-03 described — and if it does not carry that safeguard, the only responsible move is to treat the operation as non-retryable regardless of what kind of fault caused the failure, falling back to cancel-and-surface even for a fault that would otherwise clearly call for a delayed retry.

⭐ THE EARNED INSIGHT Fault classification and idempotency answer two completely independent questions, and the Retry pattern only ever looks disciplined when both have actually been checked, not when either one has been checked well. A perfectly classified fault — correctly identified as a rare blip, correctly given an immediate retry — still produces a duplicated side effect if the operation underneath was never idempotent to begin with, and a perfectly safeguarded idempotent operation still wastes latency and adds load if retried against a fault that was never going to clear in the first place. Neither question substitutes for the other, and a retry policy that has clearly reasoned through one of them can still fail entirely on the other — which is exactly the gap a scenario question is built to probe when it asks about a fault that "looks" transient on an operation that "looks" simple.

Consider an agent whose toolset includes a call to a translation model served behind an NVIDIA NIM endpoint, invoked as part of answering a user's multilingual request. Trace three consecutive failures the same tool call could produce, and the correct response to each.

Attempt 1 result: HTTP 400, body: {"error": "unsupported_language_pair",
"detail": "en-xx is not a supported target language code"}
  Classification: not transient -- the request itself named an invalid language code.
  Response: CANCEL. Retrying with the same arguments will produce the identical 400
  every time; the fix is correcting the language code, not waiting.

Attempt 1 (different call) result: connection reset, no response body at all
  Classification: transient, and looks like a one-off network-level blip -- nothing
  about the failure suggests sustained trouble, just a single dropped connection.
  Response: RETRY IMMEDIATELY. No deliberate delay; a fresh connection attempt right
  away has a real chance of succeeding, and any imposed wait would only add latency
  the situation does not call for.

Attempt 1 (a third call) result: HTTP 503, body: {"error": "service_overloaded"}
  Classification: transient, but a busy-dependency fault, not a one-off blip -- the
  NIM endpoint is explicitly reporting it cannot currently handle the request, which
  is exactly the shape of fault that adding immediate retries against would worsen.
  Response: RETRY AFTER BACKOFF. Wait 1s (+/- jitter), retry; if that also 503s, wait
  2s (+/- jitter), retry; then 4s, then 8s -- exponential growth, capped at a fixed
  maximum of 5 total attempts before giving up and surfacing the failure upstream.

Constructed scenario — the error codes and backoff numbers are illustrative, not drawn from a real NIM deployment. All three failures hit the same tool, with the same idempotency property (translation is a pure read-equivalent operation — it produces a text output from a text input with no side effect on external state, so none of the three responses needed to check for an idempotency-key safeguard at all). What differed across the three cases was entirely the fault classification, and getting that classification right — not any property of the tool itself — determined whether the correct response was to give up immediately, retry immediately, or retry on a widening schedule.

Now consider a different tool in the same agent's toolset: a call to an external weather API, which responds:

Attempt 1 result: HTTP 429, headers: {"Retry-After": "30"}, body:
{"error": "rate_limit_exceeded"}
  Classification: transient, but neither a one-off blip nor an ordinary busy-dependency
  fault -- the response carries an explicit, known cooldown rather than an unknown
  recovery time the way a generic 503 does.
  Response: RETRY AFTER DELAY, but honoring the stated 30-second window specifically,
  rather than applying a generic exponential schedule starting from 1 second. Retrying
  before the 30 seconds elapse is not "aggressive" in the overload sense -- it is
  simply guaranteed to fail again, because the server has told the caller exactly when
  the block lifts.

Constructed scenario — the specific rate-limit window is illustrative. This case is worth holding onto because it shows the fault-classification step doing real work beyond a binary transient/non-transient split: a 429 is transient in the sense that time resolves it, but a generic exponential-backoff schedule built for an unknown-duration busy fault would either retry too early (wasting attempts against a cooldown that has not expired) or, if its schedule happens to land past 30 seconds anyway, succeed by coincidence rather than because the policy actually used the information the failure handed it. A well-built retry policy reads a Retry-After signal when one is present and uses it directly, rather than substituting a one-size-fits-all schedule for information the fault already supplied.

A retry policy that gets the three-way classification right can still cause problems if a few supporting practices are skipped.

Tune the policy to the fault, not to the tool. The same tool can produce a non-transient fault on one call (bad arguments) and a transient one on the next (a momentary outage) — the classification happens per failure, not once per tool, and a policy that hard-codes one response for every failure a given tool produces will eventually apply the wrong one.

Never nest retry layers without knowing it. If a tool call is wrapped in a retry policy at the tool-integration layer, and the same call also sits inside a caller that independently retries on failure, the two policies multiply rather than add: three attempts at the inner layer inside three attempts at the outer layer is nine total attempts, not six, and each additional uncoordinated layer compounds the multiplication further. Retry logic belongs at exactly one layer, made an explicit, visible decision, not an accident of two well-intentioned pieces of code each trying to be resilient independently.

Log every retried attempt, not just the eventual outcome. A call that fails twice and succeeds on the third attempt looks, from the final result alone, identical to a call that succeeded cleanly on the first try — but the two situations carry very different information about whether a dependency is starting to degrade. Without a log of the intermediate failures, a slowly worsening fault rate is invisible until it crosses whatever threshold finally causes a user-visible failure.

Agent Development carries 15% of the NCP-AAI blueprint, tied with Agent Architecture for the heaviest weight of any domain in the exam, and objective 2.4 — error handling with retry logic and graceful failure recovery — is where this weight becomes concrete rather than conceptual. [GROUND TRUTH] (Sources/ncp-aai/domain-2-agent-development.md) states the pattern's three responses by name — "Cancel... Retry immediately... Retry after delay" — and calls out the aggressive-retry-against-an-overloaded-service trap explicitly as a warning, which signals it is a specific, testable fact rather than a general resilience platitude. The same source's framing of this domain calls the Retry and Circuit Breaker material "the most testable, most confusable" pieces of the whole domain — a signal that scenario questions here are built to test the classification step directly, not just the vocabulary.

Expect the question shape to describe a failed call's specific symptoms — a validation error, a single dropped connection, a sustained 503, a 429 with a stated cooldown — and ask which of the three responses fits, with distractors built by swapping in the wrong response for the fault described (retrying a malformed request, or canceling a fault that a stated Retry-After window makes trivially resolvable). Expect a second shape that asks about idempotency directly: a scenario names an operation (a payment, a lookup, an increment) and asks whether it is safe to retry as described, independent of how well the fault itself was classified — this is the pattern's own internal trap, testing whether a candidate remembers that fault classification and idempotency are two separate checks, both mandatory, neither substituting for the other.

Retrying immediately means reattempting a failed call with no deliberate delay, appropriate for a rare, one-off fault (a single dropped connection) where nothing suggests the underlying condition will still be present a moment later. Retrying after backoff means waiting a growing amount of time between attempts, appropriate for a busy or connectivity fault where the condition causing the failure is likely to take a real amount of time to clear, and where an immediate reattempt would add load to a dependency that is already struggling rather than give it room to recover.

Incremental backoff adds the same fixed amount of wait after every failed attempt, which under-responds to a fault that tends to need meaningfully more recovery time the longer it has already persisted — a saturated dependency five failures in is not five times as likely to have recovered as it was after one failure. Exponential backoff multiplies the wait instead of adding to it, so the delay grows to match a fault that is proving more persistent than a first failure suggested, without requiring the policy to know in advance how long any specific fault will actually last.

Yes, but only if the operation is naturally idempotent without needing one — a read, or a write that sets an absolute value rather than an incremental one, produces the same end state whether it runs once or several times, so no safeguard is required for it to be safe under any of the three retry responses. An operation that compounds with repetition (an increment, a notification send, a record creation with no dedupe check) is never safe to retry without a safeguard, regardless of how the underlying fault is classified, because the danger in that case comes from the operation's own effect, not from anything about the failure that triggered the retry.

Work through each item before checking the answer key.

1. A tool call fails with a validation error because a required field was missing from the request. What is the correct Retry response?
   • A. Retry immediately.
   • B. Cancel — the fault is not transient, and retrying the same malformed request will fail identically every time.
   • C. Retry after exponential backoff.
   • D. Retry with a longer timeout.
2. A tool call fails once due to a single dropped network packet, with no other indication of sustained trouble. What is the correct Retry response?
   • A. Cancel and surface the error.
   • B. Retry immediately, since a fresh attempt has a real chance of succeeding and no deliberate delay is needed.
   • C. Retry after a 30-second delay.
   • D. Wait for a circuit breaker to open first.
3. A tool call fails with an HTTP 503 "service overloaded" response. What is the correct Retry response?
   • A. Retry immediately, several times in a row.
   • B. Retry after a backoff delay, ideally growing (exponential), so as not to add more load to an already-struggling dependency.
   • C. Cancel — a 503 is never transient.
   • D. Retry with no delay but only once.
4. A failure response carries an explicit Retry-After: 30 header. What should a well-built retry policy do?
   • A. Ignore the header and apply a generic exponential schedule starting from 1 second.
   • B. Honor the stated 30-second window directly, since the failure already specified when retrying becomes worth doing.
   • C. Cancel immediately, since a Retry-After header means the fault is non-transient.
   • D. Retry immediately regardless of the header.
5. Why is jitter added to a backoff schedule?
   • A. To make the delay calculation faster to compute.
   • B. To prevent many callers that failed at the same moment from all retrying at the same moment again, which would recreate the original load spike.
   • C. To guarantee a retry always succeeds.
   • D. To convert an exponential schedule into an incremental one.
6. A charge_payment tool call times out with no idempotency-key safeguard in place. The fault looks like an ordinary busy-dependency 503. What is the correct Retry response?
   • A. Retry after backoff, since the fault is clearly transient.
   • B. Treat the operation as non-retryable regardless of the fault classification, because it is not idempotent and carries no safeguard.
   • C. Retry immediately, since payment services rarely fail twice in a row.
   • D. Cancel only if the amount charged exceeds a threshold.
7. Why is a backoff schedule alone not a complete retry policy?
   • A. Backoff schedules are always too slow.
   • B. A backoff schedule specifies delay between attempts but says nothing about a maximum number of attempts, so it needs an attempt cap or a handoff to a longer-horizon mechanism.
   • C. Backoff schedules only work for reads.
   • D. A backoff schedule requires an idempotency key to compute.
8. Why does nesting two independent retry layers around the same call cause a problem?
   • A. It has no effect on the total number of attempts.
   • B. The attempts multiply rather than add — three retries at one layer inside three retries at another is nine attempts, not six.
   • C. It automatically triggers a circuit breaker.
   • D. It converts transient faults into non-transient ones.

Answers

1. B. Nothing about the request's timing caused the failure; the request itself was wrong, so retrying changes nothing and cancel-and-surface is correct.
2. B. A single, isolated blip with no sign of sustained trouble is exactly the case for an immediate reattempt with no deliberate delay.
3. B. A 503 explicitly reports overload, which calls for backoff (ideally exponential) rather than immediate retries that would add to the existing load.
4. B. The failure already supplied the exact information a generic schedule would otherwise have to guess at; using it directly is strictly better than substituting an unrelated schedule.
5. B. Jitter's entire purpose is spreading synchronized retries across a window instead of a recurring instant, which a deterministic schedule alone cannot do.
6. B. Fault classification and idempotency are two separate, both-mandatory checks; a transient-looking fault does not override the absence of a safeguard on a non-idempotent operation.
7. B. Delay shape and attempt count are two different questions; a schedule with no cap can retry indefinitely against a fault that never clears.
8. B. Each layer's retry count multiplies with every other layer's, not adds, which can produce far more load against a struggling dependency than any single layer intended.

• Classify the fault before choosing a response: non-transient faults are canceled and surfaced, rare one-off blips are retried immediately, and busy or connectivity faults are retried after a growing delay.
• Exponential backoff, not incremental, is the usual choice for connectivity and overload faults, because the delay needs to grow to match a fault that is proving more persistent than a single failure suggests.
• Jitter exists specifically to prevent synchronized retry waves across a fleet of callers that all failed at the same moment.
• A Retry-After or similarly explicit cooldown signal should be honored directly rather than replaced with a generic backoff schedule that ignores information the failure already provided.
• Idempotency is a separate, mandatory check from fault classification — an operation must be naturally idempotent, or protected by an idempotency-key safeguard from M2-03, before any of the three retry responses that involve trying again are allowed to run.
• A backoff schedule needs an attempt cap; without one, a fault that never clears keeps a policy retrying indefinitely, with no built-in signal telling it to stop.

Retry alone answers what to do about a fault that is still plausibly temporary — it says nothing about what to do once a fault has clearly stopped behaving that way, retrying past any reasonable window while a dependency shows no sign of recovering.

Next: M2-05, already covering exactly that question, contrasts this pattern against the Circuit Breaker's three-state machine and works through when a resilience layer should stop retrying altogether and let the breaker take over instead.