API caching strategies for the agent era: what still holds when agents are the caller

Caching used to be an optimisation. When agents became the dominant API caller, it turned into a correctness problem.

This article works through the caching strategies that hold up when the caller is an autonomous agent making dozens of tool calls per task: HTTP caching headers, the cache-aside pattern, write-through and write-behind, and — the hard part — invalidation. Pontil's view in one sentence: most API caching guidance assumes a human refreshes the page when things look stale; agents don't, so the strategy has to guarantee freshness at the layer that serves them.

The five sections that follow: why agent traffic breaks classic caching assumptions, how HTTP caching headers actually behave in agent pipelines, the cache-aside pattern and where it fails, invalidation as the real hard problem, and how caching interacts with the tools layer that sits between agents and your API.

Why agent traffic breaks the assumptions caching was built on

HTTP caching was designed for browsers. A user loads a page, the browser caches assets, and if something looks wrong the user hits refresh. Every layer of the caching stack — browser cache, CDN, reverse proxy, application cache — was tuned for that loop. Human tolerance for staleness is high, and the correction mechanism is a keystroke.

Agents change three things at once. First, traffic patterns invert. Agent tasks routinely fan out into dozens of tool calls in a few seconds, many of them re-reading the same resource to check state after a write. That looks like a hot-loop bug to a rate limiter and like cache-friendly traffic to a CDN — and both are wrong. Second, agents can't tell stale from fresh. If your cache returns a 30-second-old inventory count, a human notices when the checkout fails; an agent confidently books the sale and moves on. Third, the read-your-writes expectation is now hard. Agents write, immediately read, and branch their reasoning on the result. Eventual consistency stops being a background concern and becomes the reason your agent loops forever.

The practical consequence: caching strategies that were fine for a public-facing REST API — long TTLs, permissive Cache-Control headers, best-effort invalidation — will produce silently wrong agent behaviour. Not slower. Wrong.

What HTTP caching headers actually do in an agent pipeline

HTTP caching headers still matter, but you need to know which layer is honouring them. Between an agent and your origin there are usually four caches: the agent runtime's own response cache (if any), a shared HTTP client cache in the agent framework, a CDN or reverse proxy, and your application-level cache. Each one interprets Cache-Control slightly differently, and the failure modes are cumulative.

The headers that matter for agent traffic:
‍

The Vary: Authorization point is the one most teams get wrong — not because standards-compliant caches are broken, but because agent stacks often introduce non-standard ones. RFC 9111 forbids shared caches from reusing a response to an Authorization-bearing request unless directives like public or s-maxage explicitly allow it, and mature shared caches (Varnish, nginx, most CDNs) honour that by default. The problem is one layer up: several agent-runtime HTTP caching libraries have shipped with broken Vary handling (see the axios-cache-interceptor advisory GHSA-x4m5-4cw8-vc44 and the Bunny CDN caching disclosure), and standard connection-pooling clients don't cache at all until you bolt an opt-in caching layer onto them — at which point the correctness of that layer is on you. If you're relying on a client-side cache anywhere in your agent stack, verify it keys on Vary and on the auth header specifically. This is the same identity boundary problem we cover in agent identity vs user identity: the moment you collapse who the caller is, caching becomes a data-leak vector.

ETag-based conditional GETs are the sleeper win. They let an agent poll the same resource cheaply — the server returns 304 Not Modified and the agent knows its cached copy is still current. Cost is a round-trip and a hash comparison, not a full response body. For high-frequency polling loops (which agents do constantly), this is the difference between a rate-limit blowout and a well-behaved caller.

The cache-aside pattern, and where it breaks under agent load

Cache-aside is the default application-level caching pattern: the application checks the cache first, falls back to the database on miss, and populates the cache on the way back. Simple, well-understood, and the source of most agent-era caching bugs.

The pattern in shorthand:

1. Agent requests resource X.
2. Application checks cache. On hit, return. On miss, query the database.
3. Application writes result to cache with TTL.
4. Application returns result.

Three failure modes matter when agents are the caller.

Thundering herd on cold cache. An agent task fans out to 30 tool calls in parallel. If none of them are cached, all 30 hit the database simultaneously. Classic problem, well-known fix (request coalescing / singleflight) — but the fan-out shape of agent traffic makes it far more common than in browser-driven workloads. If your cache-aside implementation doesn't coalesce concurrent misses for the same key, agent load will find that gap fast.

Write-then-read races. Agent writes to /orders/123, then 200ms later reads /orders/123 to check status. If your write path doesn't invalidate the cache entry synchronously — or if there's a read replica lag between the write and the cache refresh — the agent reads a stale value and makes a wrong decision. This is the read-your-writes problem, and it's why teams end up either bypassing the cache for authenticated user reads (killing the hit rate) or moving to write-through.

Negative caching that outlives the cause. Cache-aside implementations often cache 404s to protect the database. An agent creates resource X, immediately reads it, and gets a cached 404 from the read it did 200ms earlier during a validation step. The negative cache TTL was 60 seconds; the agent gave up after two retries. This shows up in error handling loops and looks like an intermittent bug — we've covered the broader retry pattern in AI agent error handling.

When cache-aside doesn't fit, the alternatives are write-through (write goes to cache and database in the same operation, cache is always fresh) and write-behind (writes buffer in the cache and flush to the database asynchronously). Write-through solves read-your-writes at the cost of write latency. Write-behind is fast but risks data loss and adds a whole class of failure modes agents can't reason about. For agent-facing APIs, write-through is usually the right trade-off on endpoints where correctness matters more than a few extra milliseconds on the write path.

Cache invalidation is the actual hard problem

Phil Karlton's line about the two hard things in computer science was funny in 1996 and expensive in 2026. Cache invalidation gets harder in the agent era for a specific reason: the set of things an agent might read after a given write is much larger than the set a human UI would read, and the coupling is invisible.

When a human updates an order in a UI, the frontend knows which views to refresh — it wrote the code. When an agent updates an order via PATCH /orders/123, downstream tool calls might read /customers/456/orders, /inventory/sku-789, /reports/daily-revenue, or a dozen other derived resources. Every one of those has its own cache entry, and every one is potentially stale.

Four invalidation strategies, in order of increasing cost and correctness:

TTL-only. Set short TTLs, accept staleness up to the TTL. Cheap, simple, wrong for anything an agent branches on. Fine for slow-moving data (product catalogues, config), dangerous for anything transactional.

Explicit invalidation on write. Every write handler enumerates the cache keys it invalidates. Correct when maintained, but the maintenance burden is real — every new endpoint, every new derived view, every new denormalised field is another place to remember to invalidate. The bug pattern: a new endpoint gets added, no one updates the invalidation list, and stale reads start appearing weeks later.

Event-driven invalidation. Writes emit events (via an outbox pattern or CDC stream), and cache invalidators subscribe. Decouples the write path from knowledge of every cache. Adds infrastructure — a message bus, consumer lag monitoring — but scales cleanly across services. This is where most teams end up once cache-aside starts breaking.

Versioned cache keys. Every resource has a version (a monotonic counter, a hash, a Last-Modified timestamp). Cache keys embed the version. Writes bump the version; old cache entries become unreachable and expire naturally. Elegant, but requires that every cache lookup starts with a cheap version fetch — which itself needs to be fast and consistent. Pairs well with ETag on the HTTP layer.

In practice, agent-facing APIs use a mix: TTL for slow-moving reference data, event-driven invalidation for anything transactional, and ETags at the HTTP layer to let the agent runtime revalidate cheaply. The one thing that doesn't work is hoping short TTLs cover for missing invalidation logic. Agents will read faster than your TTL, and they will branch on stale data before you notice.

Caching in the tools layer, not just the API

There's a caching layer most API design guidance ignores: the tools layer between the agent and your API. When an agent calls a tool, the tool definition is fetched, arguments are validated, the underlying API call is made, and the response is transformed into something the model can consume. Every one of those steps is a candidate for caching, and the right answer isn't always "cache at the HTTP layer."

Tool definitions themselves benefit from caching. Fetching the schema for 200 tools on every agent invocation is wasteful — context window management already argues for pruning tool definitions aggressively, and caching the ones you keep is straightforward. What matters is invalidation when the underlying API changes: if your OpenAPI spec drifts and the tool definition doesn't, agents will call endpoints with wrong argument shapes. We covered the drift half of this problem in OpenAPI spec drift; the cache-invalidation half is that tool definitions need to invalidate on spec change, not on a TTL.

Response caching in the tools layer is where the correctness question gets sharpest. A tool that wraps GET /orders/{id} could cache the response for 30 seconds and cut database load meaningfully. But the tools layer has more context than the HTTP layer does: it knows which agent is calling, which user's credentials are in use, and which other tools the agent has already called in this task. That context lets a tools layer make caching decisions the HTTP layer can't. It can invalidate a specific user's cached orders when that same user's agent just did a write, without touching anyone else's cache. It can bypass the cache when the agent has flagged the task as read-critical. It can serve stale-while-revalidate for reads that are part of exploration and force fresh reads for reads that precede a write.

This is the shift API caching guidance hasn't caught up with. The caller knows more about what the read is for than any HTTP header can encode. Caching decisions belong closer to that context — which usually means the tools layer, not the CDN.

What changes next, and what to measure now

The caching strategies in this article aren't new. What's new is that the caller is now an autonomous system that reads faster, branches on freshness, and can't tell when it's been served stale data. That changes the cost of every caching decision: a hit rate that looked healthy against browser traffic can hide correctness bugs against agent traffic.

Three things worth measuring on agent-facing APIs, starting today. First, stale-read incidents — how often does an agent read a value that was invalidated but not yet evicted, and what did the agent do with it? Most teams don't measure this because they can't; instrumenting the tools layer to log cache-hit-plus-recent-write is the cheapest way to start. Second, cache-miss fan-out — what does a cold cache do to your database under realistic agent concurrency? Load-test with agent-shaped traffic (bursty, correlated, per-user) not browser-shaped traffic (steady, independent). Third, invalidation lag — the time between a write completing and every derived cache entry reflecting it. If that number is longer than your typical agent task, your agents are making decisions on data they just invalidated.

The teams shipping agent projects that hold up in production aren't the ones with the most sophisticated caching. They're the ones who moved the caching decision to the layer that has the context to make it correctly, and who instrumented the failure modes hard enough to see staleness before their customers did.