Circuit breaker pattern for APIs: a practical guide for agent-era systems

By the end of this guide you'll have a working circuit breaker around an outbound API call, with the three states wired up, sensible thresholds, and the observability hooks you need to know when it trips. The examples use Python and a generic HTTP client, but the pattern ports cleanly to Node, Go, or anything with a decent async story.

Prerequisites: a service that makes outbound HTTP calls to at least one third-party or internal API, basic familiarity with retries and timeouts, and a metrics backend you can point at (Prometheus, Datadog, OpenTelemetry — anything that accepts counters and gauges).

Time required: about 45 minutes for a first working implementation, another hour to tune thresholds against real traffic.

Why this matters now: agents fan out API calls in ways human users never did. One flaky downstream can cascade across hundreds of concurrent tool invocations in seconds. Retries without a circuit breaker make the outage worse. This is one of the core API resilience patterns you need before agents go to production.

Step 1 — Pick the library, don't write the breaker yourself

The circuit breaker pattern is well-understood and well-implemented. Writing your own means re-deriving edge cases — half-open concurrency, clock skew, thread-safety — that the established libraries already handle.

Pick one of these:

• Python: pybreaker or circuitbreaker
• Node: opossum
• Go: sony/gobreaker
• Java: Resilience4j
• .NET: Polly

Install pybreaker for the examples below:

pip install pybreaker



Expected output: package installed, no errors. If you're inside a service mesh (Istio, Linkerd) you may already have circuit breaking at the proxy layer — check before adding application-level breakers. Both is fine; ignorance of both is not.

Step 2 — Wrap a single outbound call

Start with the smallest possible scope: one function, one downstream API, one breaker instance. Don't try to wrap every call in the codebase on day one.

import pybreaker
import requests

billing_breaker = pybreaker.CircuitBreaker(
   fail_max=5,
   reset_timeout=30,
   name="billing_api",
)

@billing_breaker
def fetch_invoice(invoice_id: str) -> dict:
   response = requests.get(
       f"https://api.billing.internal/invoices/{invoice_id}",
       timeout=2.0,
   )
   response.raise_for_status()
   return response.json()



Three things to notice. fail_max=5 means five consecutive failures opens the breaker. reset_timeout=30 means it stays open for 30 seconds before trying again. name="billing_api" is what shows up in metrics — name it for the downstream, not the calling function.

Expected behaviour: under normal traffic, the decorator is a no-op. Under sustained failure, the sixth call raises CircuitBreakerError immediately instead of waiting for the timeout.

Step 3 — Understand the three states

A circuit breaker has three states. Knowing them is the difference between tuning the breaker and guessing.
‍

The half-open state is the part most homegrown implementations get wrong. You don't reopen the floodgates after the timeout — you send one request, see what happens, and decide. If you send all queued traffic at once you'll hammer a recovering downstream right back into the ground.

Pybreaker handles this correctly by default. Confirm by inspecting billing_breaker.current_state while you exercise the breaker in a test.

Step 4 — Set thresholds against real data, not vibes

Defaults are a starting point, not a destination. The right thresholds depend on the downstream's actual failure profile.

Pull a week of request logs for the API you're protecting. Look for:

• Baseline error rate. If it's already 2%, fail_max=5 will trip constantly under normal load. Use a percentage-based breaker instead of consecutive-failure: pybreaker's fail_max works for low-volume calls; for high-volume traffic look at a sliding-window implementation like Resilience4j's failureRateThreshold.
• P99 latency. Your timeout (Step 2 set it to 2 seconds) must be longer than P99 under load but shorter than your caller's patience. Two seconds is fine for an internal service; for a chatty third-party it might need to be five.
• Recovery time. How long does the downstream typically take to come back? reset_timeout should be slightly longer than that. Thirty seconds is a reasonable default; thirty milliseconds is not.

Document the numbers you picked and why. The next engineer to touch this — possibly future you at 3am — needs to know the reasoning, not just the values.

Step 5 — Add a fallback, not just a failure

A breaker that only raises exceptions has moved the problem one layer up. The calling code still has to decide what to do. Give it something to do.

def fetch_invoice_safe(invoice_id: str) -> dict | None:
   try:
       return fetch_invoice(invoice_id)
   except pybreaker.CircuitBreakerError:
       # Breaker is open. Return cached or degraded response.
       return invoice_cache.get(invoice_id)
   except requests.RequestException:
       # Actual downstream failure (counted by the breaker).
       return None



Three options for the fallback, in rough order of preference:

1. Serve from cache. Stale data beats no data for most read paths.
2. Return a degraded shape. A partial invoice with a status: "unavailable" flag is usable; a 500 is not.
3. Queue the work. For writes, push to a durable queue and process when the breaker closes. This pairs naturally with the webhook reliability patterns you already need.

What you should not do: silently retry. Retrying inside a breaker fallback defeats the breaker.

Step 6 — Wire up observability

A breaker you can't see is a breaker you can't tune. Emit metrics on every state change and every call outcome.

import logging
from prometheus_client import Counter, Gauge

breaker_state = Gauge(
   "circuit_breaker_state",
   "0=closed, 1=half-open, 2=open",
   ["breaker"],
)
breaker_trips = Counter(
   "circuit_breaker_trips_total",
   "Times the breaker opened",
   ["breaker"],
)

class MetricsListener(pybreaker.CircuitBreakerListener):
   def state_change(self, cb, old_state, new_state):
       state_map = {"closed": 0, "half-open": 1, "open": 2}
       breaker_state.labels(breaker=cb.name).set(state_map[new_state.name])
       if new_state.name == "open":
           breaker_trips.labels(breaker=cb.name).inc()
           logging.warning(f"Circuit breaker {cb.name} opened")

billing_breaker.add_listener(MetricsListener())



Alert on three things:

• Breaker open for more than N minutes. A short open state is the breaker doing its job. A long one is an incident.
• Breaker flapping (more than 3 state changes in 5 minutes). Means thresholds are wrong or the downstream is genuinely unstable.
• Fallback rate above X%. Tells you how often users are seeing degraded responses.

This is also where agent traffic differs from human traffic. Agents will retry the same failed tool call across hundreds of sessions in parallel. Without per-breaker visibility you can't tell whether one downstream is broken or your whole tools layer is on fire. See AI agent observability for the broader picture.

Step 7 — Scope breakers correctly across microservices

One breaker per downstream dependency, not one per caller. If service A and service B both call the billing API, they need to share signal about its health — but they don't share a process, so they can't share a breaker instance.

Two patterns work:

• Per-service breaker, same configuration. Each service runs its own breaker against billing, configured identically. Simple. Failures take longer to propagate.
• Sidecar or service-mesh breaker. Istio, Linkerd, and Envoy all support circuit breaking at the proxy. Shared signal, no application code. Harder to add fallback logic.

For most teams, per-service breakers are the right starting point. Move to mesh-level breakers when you have enough services that maintaining identical configs becomes a chore.

One thing to avoid in microservices specifically: don't put a breaker on every internal hop. A request that crosses five services with five breakers configured at fail_max=5 can fail in dozens of confusing ways. Breakers on edges (calls leaving your trust boundary) and on known-flaky internal dependencies. Not everywhere.

Common pitfalls

Wrapping the wrong scope. A breaker around database.query() will trip on a bad query. Scope to network boundaries, not function boundaries.

Treating timeouts as failures only sometimes. A timeout is a failure. If your breaker library distinguishes timeouts from exceptions, configure both to count.

Forgetting to test the open state. Use a fault-injection tool or just point the breaker at a deliberately broken URL in staging. If you've never seen your fallback fire, you don't know if it works.

Sharing breakers across tenants. If one customer's webhook URL goes down, you don't want every other customer's calls failing fast too. Scope breakers per tenant for multi-tenant systems, or at least per customer-segment.

Stacking retries inside the breaker. Retry policies (exponential backoff) and circuit breakers solve different problems. Run them in this order: retry with backoff → circuit breaker → fallback. Not the other way around.

Tuning by gut. Every threshold in your breaker config should be traceable to a number from production traffic. "Felt about right" is how you get pages at 3am.