The orchestrator obsession is hiding the real bottleneck

The agent industry is fixated on the wrong layer. Every other launch is a new orchestrator — a smarter planner, a better router, a more elegant graph. Meanwhile, agents in production keep failing for the same reason they failed two years ago: they can't reach what the product can actually do. The orchestrator is not your bottleneck. The tools layer is. And until the industry stops conflating the two, agent projects will keep stalling at the same point.

The orchestrator is the easy part

An agent orchestrator decides what to do next. Orchestrators aim to plan, route, retry, and hold state, with varying degrees of built-in support across frameworks. That's a real engineering problem, and frameworks like LangChain, CrewAI, and the OpenAI Agents SDK have made it dramatically less painful than it was. You can stand up a competent orchestration loop in an afternoon.

But think about what happens after the loop decides what to do. The agent calls a tool. That tool either works against your product, or it doesn't. If it doesn't — because the API doesn't expose the action, or the auth model is wrong, or the contract drifts silently next sprint — none of the orchestrator's cleverness matters. You can swap planners, change models, tune prompts. The agent still can't book the meeting, refund the order, or update the record.

The orchestrator is the easy part because it operates entirely on systems the agent team controls. The hard part is everything underneath it.

Orchestrator vs tools layer: the distinction the industry blurs

The agent stack has at least four layers worth naming separately:
‍

Most "AI orchestration layer" pitches collapse the tools column into the orchestration column. They assume tools are a solved problem — that you'll bring your own, or that a generic HTTP tool plus an OpenAPI spec will do. That assumption is what's killing agent projects.

The tools layer is its own problem with its own constraints. Tools have to be discoverable, parameterised correctly for an LLM caller, executed as the authenticated user, observable when they fail, and maintained as the underlying product changes. Orchestrators don't do any of that. They're consumers of the tools layer, not providers of it.

Why orchestrator-first thinking keeps producing stalled projects

We've watched this pattern repeat across SaaS teams building agents on their own platforms. The team picks an orchestration framework, models the agent's reasoning, builds a slick demo, and gets a green light. Then they go to wire the agent into the actual product and discover three things at once:

1. The internal APIs cover a fraction of what the UI can do. Across the SaaS engagements we've run at Pontil, internal APIs have consistently exposed well under half of the actions a user can take through the product. The rest lives in form handlers, server actions, and code paths that were never meant to be called from outside.
2. The APIs that do exist were designed for first-party UI consumption. They take session cookies, return HTML fragments, or assume context the agent doesn't have.
3. Nobody owns the work of closing that gap. Platform engineering is busy. Product engineering ships features. The agent team has no mandate to rewrite the API layer, and even if they did, it's a multi-year programme.

None of those problems live in the orchestrator. You cannot prompt your way past them, and you cannot framework your way past them. They're access problems, not reasoning problems. The orchestrator was the wrong place to start because it was never the binding constraint.

The industry's enthusiasm for orchestration is partly because orchestration is genuinely interesting and partly because it's the layer venture-funded teams can ship without coordinating with anyone. The tools layer requires touching the product. That's harder, less glamorous, and exactly where the value is.

What an honest agent stack looks like

If you take the orchestrator off its pedestal, the picture rearranges. The reasoning layer is a commodity — the foundation model providers are competing it down to the floor. The orchestration layer is converging on a handful of patterns; pick a framework, move on. The product layer already exists; it's where your customers and revenue live.

The layer that actually decides whether your agent works in production is the one between orchestration and product: the tools layer. It has to do four things, and most teams underestimate all four.

• Generate tools that match the product. Not a generic HTTP wrapper. Tools whose shapes, names, and parameter semantics map to what the product actually does — including the parts the public API never exposed.
• Run them reliably. Deterministic invocation, rate limiting, auth lifecycle, observability when something goes wrong at 3 a.m.
• Execute as the authenticated user. Not a shared service account. Permissions, data visibility, and audit trails have to honour the real identity, or you've just built a compliance incident waiting to happen.
• Stay current as the product changes. The product ships every week. If your tools don't track those changes automatically, you're back to bespoke connector maintenance — and bespoke connectors don't compound.

Notice how little of that is the orchestrator's job. Notice also how little of it any orchestration framework attempts to solve. The frameworks are honest about their scope; the industry conversation around them is not.

Stop calling the tools layer an implementation detail

The deepest problem with orchestrator-first framing is that it treats the tools layer as plumbing. Just hook up the APIs. Just expose the functions. Just write the wrappers. That language hides a multi-quarter, multi-team programme behind a verb.

If you've worked on a real agent project, you already know which layer ate the timeline. It wasn't the planner. It wasn't the model. It was the months spent figuring out how to get the agent to do the thing — the thing that any user could do in the UI in three clicks, but that no API exposed, and that nobody had time to build a connector for, and that would break next sprint anyway when the underlying product shipped a refactor.

That's not an implementation detail. That's the thing that decides whether the project ships.

How Pontil fits

We started Pontil because the orchestrator-vs-tools distinction wasn't an academic one — it was the difference between agent projects that shipped and agent projects that died in pilot. We're a Tools-as-a-Service platform: we generate, run, and maintain the tools agents need to reach what your product can actually do, working against the APIs and code you already have rather than waiting for a rewrite.

That means your orchestrator choice stops mattering as much as it currently does. Pick whichever framework fits your team. The tools layer underneath it is what we hold. If you want to see how that changes the shape of an agent project, the demo is the fastest way in.

What to do differently on Monday

Stop benchmarking orchestrators against each other and start asking a harder question: of the things your agent needs to do, how many can it actually reach today? Not in the demo. In production, as the authenticated user, with the auth, rate limits, and observability your security team will sign off on. Whatever percentage you land on is the real ceiling on your agent project. The orchestrator only operates on top of that ceiling. Raising it is the work that matters, and it lives in a different layer than the one most of the industry is talking about.