AI agent error handling: a practical guide to retries, circuit breakers, and recovery

By the end of this guide, you will have a working error-handling layer for an AI agent that calls tools against real APIs. You will know which errors to retry, which to surface, when to trip a circuit breaker, and how to keep an agent from looping on the same failure until it exhausts its context window.

Prerequisites: a running agent that calls at least one tool over HTTP, structured logging on tool calls, and the ability to change the code path between the agent and the tool. Time required: about an hour to read and adapt, longer to roll out through your own review.

Step 1 — Classify every tool call error before you touch retry logic

Most broken agent error handling starts here. Teams wire in retries first, then discover the agent is politely re-sending malformed requests forty times. Classification comes before retry policy.

Sort every error your tool layer can produce into four buckets:
‍

Do the classification at the tool-runtime layer, not inside the model prompt. Models are inconsistent classifiers of HTTP responses under load. Give them a decision that is already made.

Action: write a classify_error(response, exception) function that returns one of transient, rate_limited, client_error, or auth_error. Every tool call routes its failure through it before anything else runs.

Step 2 — Retry transient errors with jittered exponential backoff

Retries only apply to transient and rate_limited. Never retry client_error or auth_error — those are decisions for the model or the user, not the runtime.

For transient errors, use exponential backoff with full jitter. Fixed delays synchronise retries across concurrent agent runs and hammer the upstream service exactly when it is recovering.

import random, time

def backoff_delay(attempt: int, base: float = 0.5, cap: float = 30.0) -> float:
   # Full jitter: sleep = random(0, min(cap, base * 2 ** attempt))
   return random.uniform(0, min(cap, base * (2 ** attempt)))

def call_with_retry(fn, max_attempts: int = 4):
   for attempt in range(max_attempts):
       try:
           response = fn()
           kind = classify_error(response, None)
           if kind is None:
               return response
           if kind == "rate_limited":
               time.sleep(parse_retry_after(response))
               continue
           if kind == "transient":
               time.sleep(backoff_delay(attempt))
               continue
           return response  # client_error, auth_error — bubble up
       except NetworkError:
           if attempt == max_attempts - 1:
               raise
           time.sleep(backoff_delay(attempt))
   return response



Cap attempts at 4. Beyond that, the fault is either the upstream service or your classifier, and more retries only extend the outage inside the agent turn. LLM agent retry patterns that loop past 4 attempts are usually masking a bug in step 1.

Action: wrap every outbound tool call in call_with_retry. Log attempt, kind, and total elapsed time per call. You need those fields in step 6.

Step 3 — Honour idempotency before you enable retries at all

Retries are unsafe on non-idempotent operations. If the tool creates a resource, charges a card, sends a message, or triggers a workflow, a retry after a network timeout can double-execute the action even though the agent thinks the first call failed.

Before turning on retries for any write operation, add an idempotency key. Most modern APIs accept one via header (Idempotency-Key) or body field. Generate it deterministically from the tool call arguments plus the agent turn ID, so a retry sends the same key and the upstream service dedupes.

import hashlib, json

def idempotency_key(tool_name: str, args: dict, turn_id: str) -> str:
   payload = json.dumps({"tool": tool_name, "args": args, "turn": turn_id}, sort_keys=True)
   return hashlib.sha256(payload.encode()).hexdigest()



If the API doesn't support idempotency keys, either don't retry writes, or add a check-then-act step: read the resource first to see if the previous call actually succeeded before retrying. This is covered in more depth in our practical guide to idempotency keys for agent tool calls.

Action: audit your tool list. Mark each tool as read or write. Writes without idempotency support get max_attempts=1 in step 2 until you fix them.

Step 4 — Add an agent circuit breaker per tool

Retries handle single-call failures. They don't handle a downstream service that has been broken for ten minutes. Without a circuit breaker, every agent turn keeps trying, burns tokens on error handling, and adds load to the failing service.

A circuit breaker sits per-tool (or per-tool-and-endpoint) and moves through three states:

• Closed — calls flow through, failures are counted.
• Open — calls fail fast without hitting the network, for a cool-off window.
• Half-open — a single probe call is allowed; success closes the breaker, failure re-opens it.

class CircuitBreaker:
   def __init__(self, failure_threshold=5, window_seconds=60, cool_off=30):
       self.failure_threshold = failure_threshold
       self.window_seconds = window_seconds
       self.cool_off = cool_off
       self.failures = []
       self.opened_at = None

   def before_call(self):
       now = time.time()
       if self.opened_at and now - self.opened_at < self.cool_off:
           raise CircuitOpenError()
       if self.opened_at and now - self.opened_at >= self.cool_off:
           self.opened_at = None  # half-open probe

   def record(self, success: bool):
       now = time.time()
       self.failures = [t for t in self.failures if now - t < self.window_seconds]
       if success:
           self.failures.clear()
           self.opened_at = None
       else:
           self.failures.append(now)
           if len(self.failures) >= self.failure_threshold:
               self.opened_at = now



Scope one breaker per tool, not one per agent. A failing calendar API should not block a working CRM lookup. If you have separate endpoints under one tool with different reliability profiles, scope per endpoint. This is the same circuit breaker pattern used across API integrations — the agent case just changes what happens when the breaker is open.

Action: instantiate a breaker per tool at startup. Wrap call_with_retry so before_call runs first and record runs on completion.

Step 5 — Turn every surfaced error into a structured signal the model can act on

When an error reaches the model, the model has to decide: try a different tool, ask the user, or give up. Free-text error strings make this decision worse. Error: 422 tells the model nothing useful.

Return a structured error object to the model with three fields at minimum:

• kind — one of the four categories from step 1, plus circuit_open and exhausted_retries.
• retryable — boolean. The runtime already tried. Tells the model not to just re-invoke the same tool.
• guidance — one short sentence in plain English about what the model should consider next.

Example:

{
 "kind": "client_error",
 "retryable": false,
 "guidance": "The customer ID was not found. Ask the user to confirm the ID or search by email instead.",
 "detail": "404 Not Found: customer_id=cus_abc123"
}



Models loop on tool errors when the error looks recoverable but isn't. Setting retryable: false explicitly, and phrasing guidance as a next action rather than a diagnosis, cuts the loop rate sharply. This is the same principle behind good agent guardrails at the runtime layer: make the safe path the obvious path.

Action: replace every raw exception message that reaches the model with a structured error. Do the mapping once, in the tool runtime.

Step 6 — Instrument recovery, not just failure

Most teams log the failure and move on. That misses the more useful signal: how often the agent recovered, and how.

For every tool call, record:
‍


recovered is the key one. If 30% of your tool calls succeed only after retry, you have a downstream reliability problem the model is quietly masking. That signal only exists if you distinguish success from recovered.

Action: emit one structured log line per tool call with the fields above. Route to whatever your team uses — most agent tracing setups accept structured spans natively.

Common pitfalls

Retrying on 4xx errors. A 400 will be a 400 on the next try. Retrying wastes budget and — worse — teaches the agent that bad requests are worth repeating. Classify first, retry second.

One global circuit breaker for all tools. A single flaky tool trips the breaker and every other tool fails fast for the cool-off window. Scope breakers per tool.

Backoff without jitter. Fixed backoff synchronises retries across parallel runs and creates thundering herds against a service already under stress. Full jitter is worth the two-line change.

Retrying write operations without idempotency keys. Duplicate charges, duplicate emails, duplicate tickets. Idempotency is a prerequisite for retry, not a nice-to-have.

Passing raw stack traces to the model. Long, unstructured error strings burn context and confuse the model into retrying non-retryable failures. Give it kind, retryable, guidance — nothing more.

Treating circuit_open as a failure. It's a signal to the model to route around the failing tool, not to give up. Tell it that in guidance.

Silent recovery. If your dashboards only show failures, a service degrading from 99.9% to 92% success looks fine because the retries hide it. Track recovered separately.