Agents in production
Why AI agents fail in production (and how to fix it)
Agent project stalls aren't random. There's a structural pattern behind every failure — and naming it is the first step to getting unblocked.
7 minute read
Agents in production
Agent project stalls aren't random. There's a structural pattern behind every failure — and naming it is the first step to getting unblocked.
Most AI agent projects don't fail because the model is wrong. They fail because the agent can't reach the product it's supposed to act on.
The proof-of-concept works. The model reasons well. The agent plans the right sequence of steps. Then it hits the API — and the API doesn't expose what the UI has always done. The form logic lives in the controller. The workflow spans screens. The permission check happens somewhere the public API can't see. The agent stalls.
This is the pattern. It shows up whether the company has two products or twenty, whether the agent is built on LangChain or a custom framework, whether the model is from OpenAI or Anthropic. The failure mode isn't the model. It's the access layer.
This piece maps the specific failure modes, explains why each one is structural rather than incidental, and walks through what a proper fix looks like.
Proof-of-concept agents are built against controlled surfaces. A developer picks a handful of API endpoints that work cleanly, wires them up as tools, and demonstrates the agent completing a task. It looks good. It ships to production.
Production is different. Real users want the agent to do what they do in the UI — which is almost never what the public API exposes. The gap between "what the API supports" and "what a user can do in the interface" is where agent projects go to die.
The irony is that the gap isn't new. SaaS products have always had it. APIs were built for a different era — point-to-point integrations, webhooks, developer-facing use cases. The UI kept growing. The API didn't keep pace. For human users it didn't matter: they used the interface. For agents it's fatal: the interface isn't callable.
The most common failure. The agent needs to perform a task that users complete through the interface every day — updating a record with custom field logic, running a workflow that spans multiple objects, triggering a process that involves a permission check in the controller layer. None of it is in the API.
The usual response is to try to extend the API — expose a new endpoint, create a wrapper, hack something together. This works for one task. It doesn't work for the twenty tasks the next sprint surfaces. And it doesn't work at all when the product has multiple surfaces with different tech stacks.
The structural fix is connector generation that works against the codebase, not just the published API spec. If the capability exists in the code, it can be surfaced as a tool — without rewriting the product's API layer.
Agent auth is different from human auth. A human logs in once and stays logged in through a session. An agent invokes tools repeatedly, often across long-running tasks, often as multiple users simultaneously. Token expiry, refresh logic, and credential lifecycle management — all of which a human browser handles invisibly — become explicit engineering problems.
The failure mode here is subtle. Auth works in testing, where calls happen in quick succession against a single user. It breaks in production, where a long-running agent task hits an expired token mid-sequence, or where high-volume invocation runs into rate limits on the underlying auth provider.
The right model is full auth lifecycle management at the runtime layer: obtaining, refreshing, and revoking credentials without pushing that complexity to the agent or to the product's own engineers. Tool calls should execute as the authenticated user — with the user's permissions, data visibility, and audit trail — without requiring the agent to manage credentials directly.
APIs are rate-limited for human-scale usage patterns. A human might invoke a workflow a few times a day. An agent might invoke it hundreds of times in an hour — not because it's misbehaving, but because it's doing what it was asked to do.
Most SaaS APIs weren't designed with this in mind. The rate limits that exist are there to prevent abuse, not to handle agent-scale load. When an agent project hits production, it often discovers this the hard way: the agent runs fine in testing, then gets throttled in production because it's generating request volumes the API wasn't designed for.
The fix isn't to slow the agent down. It's to handle rate limiting at the runtime layer — queuing, backoff, load spreading — so the agent can operate at its natural pace without degrading the product's underlying APIs for other consumers.
SaaS products change. APIs change with them. When a product team updates an endpoint, renames a field, or changes the structure of a response, any agent tooling built against the old shape breaks — often silently.
The failure mode is pernicious: the agent doesn't error out immediately. It continues to run, producing wrong results or incomplete actions, until someone notices something is off. By then, the drift has often accumulated across multiple changes.
The right architecture aligns tool maintenance with the product's own SDLC. When the product changes, the tools update alongside it — not as a separate, manually-triggered process, but as part of the same change cycle. Automated maintenance, triggered by the same CI that ships the product change, is what makes this scale.
Even when an API is well-documented and accessible, the tools generated from it often don't match what users actually do. Raw API actions — "create record," "update field," "fetch object" — are not the same as business tasks — "enrol a student," "process a claim," "close a deal."
An agent given raw API tools has to reason about how to compose them into the right sequence for every task. This is slow, error-prone, and wastes model capacity on orchestration work that could be done in the tool layer. It also means the agent's success rate is highly sensitive to how well the model understands the product's data model — which is not information that belongs in the agent's context window.
The fix is tool generation that composes connector actions into higher-level workflows: units of work that represent real business tasks, with the sequencing and logic encoded in the tool rather than delegated to the model. The people who know the product best — the engineers and product managers who built it — should be in the loop defining and approving what gets exposed.
The failure modes above all point to the same structural gap: agent tooling is an afterthought, bolted onto an API layer that was never designed for it.
A production-ready stack has four properties:
Coverage. The tools reflect what the product can actually do — not just what its published API exposes. Connector generation works against the codebase, not just the spec.
Lifecycle management. Auth, rate limiting, and failure handling are managed at the runtime layer. The agent and the product's engineers don't have to think about them.
Maintenance. Tools stay in sync with the product as it changes. Updates are triggered by the same change cycle that ships the product, not by a separate manual process.
Task alignment. Tools represent real business tasks, not raw API actions. The sequencing and logic are encoded in the tool layer, not delegated to the model.
Building this from scratch is a multi-year project. Most engineering teams facing an agent project don't have that time — and can't justify the investment when the product roadmap already has a full queue.
Agent projects stall at the same point because the problem is the same: the product's API layer wasn't built for agent-scale, agent-pattern access. The failure modes are predictable. So is the fix.
The question isn't whether to solve the access problem. Every team that wants agents to do real work has to solve it. The question is whether to solve it in-house — slowly, expensively, and in a way that doesn't compound across products — or to treat it as the infrastructure problem it is and get it out of the agent project's critical path.
Agents that can actually act on the products they're supposed to act on. That's the starting point. Everything else is model selection.
Stay up to date on the ever changing agentic landscape.
GET STARTED
Start with a 30-minute discovery call
contactus@pontil.com
