Capability discovery for AI agents: how they find tools, and why most of them don't

How does an agent know what it can do? The answer sounds trivial until you build one. A model doesn't intuit that your product has a create_invoice endpoint. It doesn't guess that the CRM it's supposed to update has a notes field or that the notes field is capped at 4,000 characters. Something has to hand it that information, in a form it can reason over, at the right moment in the loop. That something is capability discovery — and it's one of the least-discussed load-bearing pieces of the whole agent stack.

The view here is simple. Most agent projects treat capability discovery as a solved problem because the model saw a list of tools in the system prompt. It isn't solved. It's deferred. As tool counts grow past a few dozen, as products change underneath the agent, and as different users get different permissions, the naive approach — dump every tool schema into the context window — falls apart in ways that look like model failure but are actually a discovery-layer failure.

This piece walks through what capability discovery actually is, the four patterns teams use in production, where each one breaks, and what the emerging picture looks like as the tools layer matures.

What capability discovery actually means

Strip away the terminology and capability discovery answers three questions on every turn of the agent loop:

1. What can I do right now?
2. For each of those things, what inputs does it take, what does it return, and when should I use it?
3. Am I allowed to do it as this user, in this context?

The first is the classic "tool list" problem. The second is the tool description problem — which we've written about separately, because descriptions are where a lot of selection errors actually originate. The third is where discovery collides with authorisation, and it's the piece most implementations skip.

In a foundation-model tool-calling loop — Anthropic's Claude, OpenAI's Responses API, Google's Gemini — the mechanism looks the same. The client sends a list of tool definitions with the user message. The model returns either a text reply or a structured tool call. The client executes the call, returns the result, and the loop continues. Discovery is what fills that list of tool definitions in step one. Get it wrong and everything downstream is guessing.

The naive implementation works fine for demo agents with five tools. Real SaaS agents have hundreds of possible actions. That's where the four patterns diverge.

The four patterns teams actually use

Each one is a real answer to a real problem. None of them is complete.

Static injection: fine until it isn't

The first agent you build will use static tool injection. You hand-write a list of five or ten tools, embed them in the system prompt or attach them to each request, and the model picks between them. This works. It works well up to somewhere between 20 and 50 tools, depending on the model and how well-written your descriptions are.

Past that point, three things go wrong. Selection accuracy degrades — the model picks the wrong tool, or invents parameters, or fixates on the first tool that looks close. Context cost climbs. And any change to the product means editing prompts and redeploying. We've covered the tool-count ceiling in detail in this piece on how many tools an agent can actually handle — the short version is that the ceiling is real, it's a selection-accuracy ceiling more than a context-window ceiling, and it hits earlier than people expect.

Static injection also can't handle per-user variation. If Alice can approve refunds and Bob can't, the tool list either lies to Bob (offering him a refund tool that will fail at runtime) or you build a branching prompt system that regenerates the list per user — which is just runtime discovery, badly implemented.

Runtime tool listing (the MCP shape)

The Model Context Protocol (MCP) formalises what teams were already building ad hoc: a server that the agent connects to, and a list_tools call that returns the available tools at session start. Anthropic shipped MCP as an open protocol in late 2024. The pattern predates it — every agent framework had some version of "call the server to find out what's available" — but MCP gave it a name and a wire format.

The move from static to runtime is real progress. The tool list can change without a client redeploy. The server can filter by authenticated user before returning the list, so Bob never sees the refund tool. Auth, versioning, and deprecation move from prompt engineering to server-side concerns.

What MCP doesn't fix is the underlying scale problem. The server can send back 300 tools in a list_tools response, and the model still has to reason over 300 tools in its context. Runtime discovery moves the source of the tool list; it doesn't shrink the list. If you have hundreds of capabilities across a multi-product suite, runtime listing alone isn't enough. We've covered what MCP does and doesn't fix — capability discovery at scale is squarely in the "doesn't fix" column.

There's also a client-side problem MCP doesn't address: how does the agent know which MCP server to connect to in the first place? That's a discovery layer above the protocol, and it's currently solved by configuration files, hand-maintained catalogues, and the occasional public directory.

Retrieval-based discovery

When the tool count is too large to fit in context, teams reach for retrieval. Embed every tool description in a vector store. On each user turn, embed the query, search for the top-K most similar tool descriptions, and send only those to the model. Now the agent has effective access to thousands of tools without paying the context cost of listing them all.

This works and it's underused. But it introduces a new class of failure: retrieval miss. If the user asks "cancel my subscription" and the relevant tool is called update_billing_status, the semantic match may not surface it. If two tools are near-duplicates in description, the wrong one comes back first. If the user's intent is multi-step ("find overdue invoices and send reminders"), a single retrieval query may only surface one half of the workflow. We wrote about these failure modes specifically in the context of agent tool selection — retrieval doesn't remove selection errors, it moves them from the model to the retrieval index.

The fix is to treat retrieval as a first-pass filter, not the final answer. Retrieve 20, let the model pick from 20, and design tool descriptions specifically for retrievability (canonical action verbs, synonyms in the description body, no marketing language). Retrieval-based discovery is closer to a search-engine problem than an agent problem, and teams that treat it that way get better results.

Agent cards and manifests

The fourth pattern is the one furthest from settled. The idea, borrowed roughly from .well-known conventions and OpenAI's earlier ChatGPT plugin manifest, is that a product publishes a document at a predictable URL describing what its agent-accessible capabilities are, how to authenticate, and where to call. The agent (or the platform it runs on) fetches this document, parses it, and knows what the product offers before any session begins.

Google's Agent2Agent (A2A) protocol calls this an "agent card". Several other efforts — from foundation model providers, from agent framework vendors, from standards bodies — are converging on some version of the same idea. None of them has won yet.

The appeal is that agent cards move discovery to the product's control plane. The company that owns the product decides what it exposes, publishes it, and updates it. The agent doesn't have to be pre-configured. This is the right shape for a world where agents move between products the way browsers move between websites.

The problem is that agent cards discovered so far mostly describe the same 2% of product surface that existing APIs already expose. If your public API only covers the top-of-funnel actions — create account, list users, get invoice — then your agent card describes those same actions, dressed up in new metadata. The card is honest about what's reachable; what's reachable is still the thin slice. We've written before that your APIs expose 2% of what your product can do, and no manifest format changes that number.

Where discovery collides with authorisation

Capability discovery is often described as if it were a pure metadata problem. It isn't. Every tool an agent can see is a tool it might call, and every tool it might call is a permission check that has to happen somewhere.

There are two places to enforce it. You can filter at discovery — only return tools the user is allowed to invoke. Or you can filter at execution — let the agent see everything and reject unauthorised calls at runtime.

Filter-at-discovery is cleaner for the model. It never sees a tool it can't use, so it never tries and fails. It's also information-leaky in one direction: if Bob asks the agent "can you approve refunds?" and the tool isn't in his list, the agent has to explain "you don't have that permission" rather than "that tool doesn't exist", or Bob learns nothing about the product's real capability. That may or may not matter for your product.

Filter-at-execution is honest about the surface but wastes turns. The model tries the refund tool, gets a 403, apologises, tries something else. Every wasted turn costs latency, cost, and reliability.

Most production systems end up doing both. Filter obviously-unauthorised tools out of discovery to keep the list clean. Enforce the real permission check at execution because you can't trust a client-side filter as your security boundary. This is one of the reasons we've argued that the runtime layer is where boundaries actually hold — discovery is a UX layer, not a security layer.

The knot gets tighter when tools execute as the authenticated user rather than a shared service account. If the same nominal tool (approve_expense) is available to a manager but produces different results depending on the user's role, is that one tool or many? Discovery-layer answers vary. The correct answer is one tool with role-dependent behaviour, discovered per user, executed with the user's identity — but few current implementations get there.

What discovery does to the agent loop

It's worth being concrete about how discovery lands inside a running agent. Consider a five-turn workflow: the user asks for something, the agent picks a tool, the tool returns data, the agent picks another tool, the tool commits an action. Where does discovery run?

In the naive model, once per session. The list of tools is fetched at the start, the model reasons over it for the whole conversation. This is what most implementations do and it mostly works.

But consider what happens when the tool the agent needs on turn four wasn't relevant on turn one. If discovery is per-session and retrieval-based, the top-K tools fetched at session start may not include the one needed later. Now the agent is trying to answer a question with a tool list that no longer reflects the conversation. The fix is per-turn re-retrieval, which is expensive but often correct.

Or consider what happens when a tool is deprecated mid-session. The server knows. The client doesn't, because it fetched the list once. The agent calls the deprecated tool, gets an error, and now has to recover. This is the same connector-drift problem that iPaaS teams have wrestled with for years, arriving now in the tools layer.

The implication is that capability discovery is not a one-shot operation. It's a continuous property of a healthy agent runtime. Systems that treat it as a startup step will keep hitting the same class of bug for the same reason.

What comes next

The honest state of capability discovery in 2026: nobody has a complete answer. MCP handles the protocol. Retrieval handles the scale. Agent cards handle the cross-organisation piece. Authorisation cuts across all three. No single pattern covers the whole shape.

What's likely to change first is the retrieval story. Better embeddings, better tool description conventions, and richer per-tool metadata (input examples, expected outputs, common failure modes) are all improving faster than the protocol layer. Expect production agent teams to be doing quite sophisticated tool-index engineering by mid-2026, even if the protocols underneath stay noisy.

What won't change quickly is the underlying access problem. Discovery can only surface tools that exist. If your product's real capability lives in code paths that no API touches, no discovery format will help the agent reach it. Which brings the question back to where every serious agent conversation ends up: not "how do agents find tools?" but "why are there so few tools to find?"

The first question is a protocol problem. The second is a product problem. Teams that get the first one right and haven't started on the second will be well-instrumented agents with nothing much to do.