API rate limiting best practices for SaaS in the agent era

Rate limiting was designed for human-driven traffic. Agents don't behave like humans. They burst, they retry on failure, they parallelise tool calls, and they hit endpoints in patterns no UI ever produced. The rate limiter you wrote three years ago is now the thing breaking your agent rollout.

This guide walks through the seven steps to design a rate limiting strategy that holds up under agent traffic — covering algorithm choice, tier design, response headers, and the failure modes most teams find in production. Prerequisites: a working API, some form of authentication, and a way to identify callers (API key, OAuth client, user token). Time required: a focused half-day to design, one to two sprints to implement well.

Step 1 — Audit who's actually calling your API

Before picking an algorithm, find out what traffic looks like today. Agent traffic and human-developer traffic have different signatures, and a single global limit treats them the same when it shouldn't.

Pull seven days of access logs. Group requests by client identifier (API key, OAuth client_id, or user token). For each caller, calculate: requests per second at peak, requests per day total, and the ratio of bursts (>10 requests in one second) to steady-state traffic. Agents will show up as high-burst, high-parallelism callers. Human developers building integrations will show up as steady, low-volume callers.

# Quick log audit — adjust field positions to your log format
awk '{print $4, $7}' access.log | \
 sort | uniq -c | sort -rn | head -50



The output tells you which callers need their own tier and which can share a default. If 80% of your burst traffic comes from 5% of callers, you have a tiering problem, not a global limit problem.

Step 2 — Pick an algorithm that matches your traffic shape

There are four algorithms worth considering. Pick one based on what you saw in Step 1.
‍

For agent-era SaaS APIs, token bucket is usually the right default. Agents legitimately burst — a planning step might fan out into 20 parallel tool calls — and token bucket allows that as long as the long-run rate stays within budget. Sliding window is more accurate for fairness but costs more memory and rejects legitimate bursts that token bucket would absorb.

Fixed windows are fine for internal APIs but produce a 2x edge spike: a caller can hit the limit at the end of one window and again at the start of the next. Don't use them on anything customer-facing.

Step 3 — Implement the limiter in front of your API, not inside it

Rate limiting belongs at the edge — in your API gateway, reverse proxy, or a dedicated middleware layer. Not in application code. Putting it inside the app means every rejected request still costs you a process, a database connection check, and log volume.

Kong, Envoy, and NGINX all ship with edge rate-limiting modules, but the algorithms differ: Envoy's local_ratelimit filter is genuine token bucket; Kong's official Rate Limiting plugins use fixed- and sliding-window counters; and NGINX's limit_req is leaky-bucket (though burst with nodelay can approximate token-bucket behaviour). Pick the one whose default algorithm matches what you decided in Step 2, or layer your own. If you're on a cloud provider, AWS API Gateway and Cloudflare both expose configurable rate limits at the edge.

For a self-managed token bucket in Redis, the canonical Lua script is short:

-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill_rate, ARGV[3] = now, ARGV[4] = cost
local bucket = redis.call('HMGET', KEYS[1], 'tokens', 'last')
local tokens = tonumber(bucket[1]) or tonumber(ARGV[1])
local last = tonumber(bucket[2]) or tonumber(ARGV[3])
local elapsed = math.max(0, tonumber(ARGV[3]) - last)
tokens = math.min(tonumber(ARGV[1]), tokens + elapsed * tonumber(ARGV[2]))
local allowed = tokens >= tonumber(ARGV[4]) and 1 or 0
if allowed == 1 then tokens = tokens - tonumber(ARGV[4]) end
redis.call('HMSET', KEYS[1], 'tokens', tokens, 'last', ARGV[3])
redis.call('EXPIRE', KEYS[1], 3600)
return {allowed, tokens}



Run this as an EVAL from your gateway. It's atomic, sub-millisecond, and scales horizontally with your Redis cluster.

Step 4 — Tier limits by plan, not by endpoint

The common mistake: setting per-endpoint limits like "100 requests/minute to /search." That treats every caller the same and punishes high-paying customers running production agents.

Tier by plan, then optionally adjust per-endpoint within a plan. A workable starting structure:
‍

The concurrency cap matters more than people think. Agents parallelise aggressively. A caller within their per-minute budget can still open 200 simultaneous connections and exhaust your database pool. Limit concurrent in-flight requests separately from rate.

For pricing alignment, base limits on the unit that scales with customer value — seats, workspaces, or active users — not raw API calls. This is how rate limiting tiered pricing actually works: limits track the value the customer is getting, not the cost you're trying to recover.

Step 5 — Return useful headers and a real error code

When you reject a request, the caller needs enough information to back off correctly. The minimum response:

HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1735689600
Content-Type: application/json

{
 "error": "rate_limit_exceeded",
 "message": "Rate limit of 600 req/min exceeded. Retry after 12 seconds.",
 "retry_after_seconds": 12
}



The X-RateLimit-* headers are a de facto convention popularised by GitHub, Stripe, and others; the IETF draft-ietf-httpapi-ratelimit-headers proposes a standardised RateLimit and RateLimit-Policy replacement without the X- prefix, which is worth tracking if you're designing a new API today.

The Retry-After header is what well-behaved clients (and most agent frameworks) read to schedule backoff. Without it, agents fall back to exponential backoff with jitter, which is fine but slower to recover. With it, they retry at exactly the right moment.

Also include X-RateLimit-Remaining on successful responses, not just 429s. Agents that know they're at 5/600 budget left will pace themselves. Agents that learn their budget only by being rejected won't.

Step 6 — Decide what to count, and when

Not every endpoint costs the same. A GET /users/{id} is cheap. A POST /reports/generate might spin up a worker for thirty seconds. Counting them as one unit each penalises light callers and lets heavy callers through.

Two workable approaches:

1. Weighted costs. Assign each endpoint a token cost (1 for cheap reads, 5 for writes, 50 for expensive computes). Deduct that cost from the bucket per call. Token bucket handles this naturally — the Lua script above already takes a cost argument.
2. Separate buckets. Maintain distinct buckets for read traffic, write traffic, and expensive operations. Callers get a separate budget for each. More headers to return, but clearer to reason about.

Weighted costs are simpler to ship. Separate buckets are easier to explain to customers and easier to tune. For most SaaS APIs that need to support both human developers and agents, separate buckets win — agents tend to concentrate on specific endpoint classes, and you'll want to limit those classes without throttling the rest.

Step 7 — Monitor rejection rates and adjust

A rate limiter you don't measure is one you can't tune. Track three metrics:

• Rejection rate per caller — what percent of a caller's requests are getting 429'd. Above 5% sustained means either the caller is misbehaving or your limit is too low for legitimate use.
• Bucket utilisation distribution — for each plan, the p50, p95, and p99 of bucket-fill at request time. If p95 is consistently near zero, your limit is too tight. If p95 is consistently near full, you have headroom.
• 429 latency — how long callers wait between hitting the limit and successfully retrying. If this climbs, your Retry-After values are wrong or callers are ignoring them.

Review these monthly. The first three months after launching new limits will show patterns you didn't predict — agent traffic shapes shift as customers move from PoC to production, and your initial tier design will need adjustment.

For deeper observability on agent traffic specifically — including which tool calls are hitting limits and which aren't — see our guide on AI agent observability.

Common pitfalls

Limiting by IP address. Agents often run behind shared infrastructure (cloud functions, NAT gateways, Anthropic or OpenAI hosted runtimes). IP-based limits will either over-restrict or fail entirely. Limit by authenticated identity — user, API key, OAuth client.

Sharing a service account across users. A common pattern in early agent projects: one OAuth client for the whole agent, acting on behalf of many end-users. From your rate limiter's view, that's one caller hitting limits for everyone. Issue per-user credentials and limit per user. See our guide on OAuth for AI agents for the delegated-access pattern.

No graceful degradation. When a caller hits their limit, returning 429 is correct. Returning 500 because your rate limiter itself fell over is not. Use a local in-memory fallback (allow everything, log loudly) when Redis is unreachable. Better to ship a brief over-limit period than to take the API down.

Forgetting webhooks count too. Outbound webhook deliveries hit the receiver's rate limits, and your retry logic needs to respect their Retry-After. See webhook reliability patterns for the full picture.

Treating rate limits as the only defence. Rate limits stop volume. They don't stop a single expensive request from taking down a service. Pair them with timeouts, request size limits, and circuit breakers on downstream dependencies.

Get these seven steps right and your API will hold under agent load. Get them wrong and your first production agent customer will be the one who finds out.