Agent tool granularity: how big should each tool be, and why it decides whether your agent works

How big should an agent tool be? A single manage_account that does everything, or fifteen small tools that each do one thing? This is the question every agent project hits about six weeks in, usually after the model starts picking the wrong tool or the token budget starts hurting. The article walks through the trade-offs.

Our view: granularity is not a taste question. It's a design constraint set by the model's selection accuracy, the shape of your API, and the workflows the agent has to complete. Coarse-grained tools save tokens and reduce selection errors, but they hide capabilities and inflate parameter schemas. Fine-grained tools mirror API endpoints cleanly, but at scale the model can't tell them apart. The right answer is almost never at either extreme, and it changes as your tool catalogue grows.

We'll cover five things: what coarse-grained vs fine-grained tools actually look like, the failure modes at each extreme, the four constraints that should drive the decision, a working heuristic for sizing, and how to evolve granularity as the catalogue grows.

What coarse-grained and fine-grained tools actually look like

Start with the concrete shapes. A fine-grained tool maps roughly one-to-one to an API endpoint. create_invoice, get_invoice, update_invoice_status, void_invoice, list_invoices — five tools, each with a tight parameter schema, each doing one thing. This is what you get if you point a code generator at an OpenAPI spec and let it emit a tool per operation.

A coarse-grained tool bundles related work behind a single entry point. manage_invoice with an action parameter (create, get, update_status, void, list) and a nested params object that changes shape based on the action. One tool, five behaviours, one schema that has to describe all of them. Some teams go further — handle_billing covering invoices, subscriptions, refunds, and disputes behind one door.

Between these two extremes sits the sensible middle: tools scoped to a domain concept, not a single endpoint. create_invoice and update_invoice are separate tools, but list_invoices and get_invoice might collapse into find_invoices with a filter parameter. Granularity here follows the shape of the work, not the shape of the API.

The OpenAI and Anthropic tool-calling APIs both let you use any of these shapes. The protocol doesn't care. The model does.

The failure modes at each extreme

Fine-grained catalogues break through selection accuracy. Once you have thirty tools whose names and descriptions look similar, the model starts confusing them. get_invoice vs get_invoice_details vs retrieve_invoice_by_id — a human can see the difference; a model reasoning over similar names and descriptions in-context often can't. We covered this in depth in why the model picks the wrong tool: selection accuracy degrades non-linearly as the catalogue grows, and the failures look like model problems but are design problems.

There's also a token cost. Every tool definition sits in the context window on every turn. Fine-grained catalogues can easily consume tens of thousands of tokens per turn — MCP-style tool schemas with rich descriptions commonly run 500–1,000 tokens each, and production reports cite single connectors chewing through 40K+ tokens before the agent has done any work. That's covered in how many tools should an agent have — the practical ceiling is lower than most teams think.

Coarse-grained tools break the other way. When you collapse ten operations behind one tool with an action parameter, the parameter schema has to describe all ten shapes. You end up with oneOf schemas, deeply nested optional fields, and descriptions like "required if action is 'create' or 'update', ignored otherwise." Models handle this poorly. They pass the wrong parameters, they invent values for fields that don't apply, they hallucinate action names that aren't in the enum.

Worse: coarse tools hide capabilities. If your manage_account tool has twelve possible actions, the model has to read the entire description to find out that transfer_ownership is one of them. Tool descriptions get long, unwieldy, and expensive — and the model still misses features buried in paragraph four. We wrote about the discipline required in how to write tool descriptions for LLM agents; at some point no amount of description work fixes an overloaded tool.

The four constraints that should drive the decision

Granularity is a function of four things. Get these right and the answer usually falls out.

Selection accuracy at your catalogue size. How many tools does the agent see per turn? If it's under 15, fine-grained is usually fine. Between 15 and 40, you need to think carefully — published benchmarks (BFCL, work by Kate et al. and Gan & Sun) show meaningful degradation starting in this range. Over 40, coarse-grained bundling or dynamic tool loading becomes structural, not optional.

Parameter schema complexity. If collapsing tools produces a schema with three or more oneOf branches or six-plus optional fields whose validity depends on other fields, you've gone too coarse. The model will pass bad parameters more often than it picks the wrong tool.

Workflow shape. If a workflow requires calling five tools in a fixed sequence every time (create → get → update → notify → log), that's a signal to bundle them. If the tools are used independently across different workflows, keep them separate.

Reversibility and blast radius. Destructive or high-cost operations (delete, transfer, refund, publish) should be their own tools, always. Never bury a delete action inside a manage_x tool with an action parameter — the model will trigger it by mistake. This connects to agent guardrails: the right layer for preventing destructive mis-calls is the tool boundary, not the prompt.
‍

A working heuristic for sizing individual tools

Here's the rule we use when reviewing tool designs: one tool per verb-object pair that a user would recognise as a distinct action. Not per endpoint, not per domain — per action.

"Create an invoice" is one action. create_invoice is one tool. "Find invoices matching some criteria" is one action, even if it maps to two API calls (list + filter). find_invoices is one tool. "Void an invoice" is a distinct action with different consequences from updating one, so it's a separate tool from update_invoice, even if the underlying API uses the same PATCH endpoint.

Three tests to run against each candidate tool:

1. Can you write the tool description in two sentences without using "or"? If the description needs "either X or Y depending on the parameters," you've bundled two tools into one.
2. Is the parameter schema flat, or does field validity depend on the value of another field? Conditional fields are a strong signal to split.
3. Would a reasonable user describe this as "one thing the system does," or as "a family of things"? Family framing means split.

This heuristic works because it aligns tool granularity with how the model reasons. Models select tools by matching the user's intent against the names and descriptions of the tools presented in-context. If your tool boundaries match natural verb-object pairs, selection accuracy improves. If they don't — either because you've bundled multiple actions or split one action across tools — the model has to work harder to route correctly, and it fails more often.

The heuristic also survives contact with awkward APIs. If your API uses a single endpoint for create-or-update (an upsert), you should still expose two tools: create_invoice and update_invoice. The API's shape is an implementation detail; the tool surface should match how agents think about the work.

How granularity should evolve as the catalogue grows

Granularity isn't a decision you make once. It changes as the catalogue grows.

Under 15 tools: stay fine-grained. Selection accuracy is not your bottleneck. Clarity is. Give each action its own tool with a clean schema. Don't over-engineer.

15 to 40 tools: start clustering. Group tools by domain in your descriptions. Add domain prefixes to names (invoice_create, invoice_find, invoice_void). Consider merging tools whose parameters overlap heavily and whose actions are always used together. Watch your eval scores — the moment selection accuracy on similar-named tools drops below 90%, act on it.

40+ tools: change the architecture, not just the tools. At this scale, no amount of tool-level tuning fixes selection. You need dynamic tool loading (fetch a subset per session based on the user's request), tool retrieval (use embeddings to select the top-K candidates before the model sees them), or a hierarchical structure (a router tool that picks a sub-catalogue). These are the patterns we cover in how many tools should an agent have — past the ceiling, granularity is the wrong lever.

There's a trap here. Teams starting with a portfolio of established products often try to skip straight to coarse-grained bundling because they know they'll end up with hundreds of tools. This usually fails. Coarse tools built without first understanding which fine-grained actions matter tend to bundle the wrong things. You end up with manage_account and manage_billing that each hide fifteen actions, and the model can't work out which entry point to use either. Better to start fine-grained per product, then bundle deliberately as you observe failure patterns.

How Pontil fits

Granularity decisions are only easy in a small catalogue. Established SaaS portfolios with multiple products don't have that luxury — the tool surface has to cover thousands of actions across products, most of which today live behind UI paths that no API exposes. That's the tools-layer problem we work on.

Pontil's connector generation produces tools sized to the actions your product actually performs, not one-to-one with API endpoints. The Tool Runtime handles the composition and execution, so "create_invoice" can be a single tool from the agent's point of view even when the underlying work spans three API calls, retries, and an auth refresh. And because tools stay current as the underlying product changes, granularity decisions don't ossify — you can re-cluster tools as you learn from production eval data without a rebuild. Teams evaluating how to structure agent access across a portfolio can book a walkthrough.

What changes when the catalogue crosses 100 tools?

Every framework we've discussed assumes the model can see the full catalogue on every turn. That assumption starts breaking well before 100 tools on most models, and is fully broken by the 100–200 range — and it breaks quietly. Selection accuracy degrades before latency does, so you notice the failures before you notice the token cost. Granularity is the lever you reach for first, but at scale it's not enough.

The next lever is architectural: dynamic loading, retrieval-based selection, or a two-stage router. These change the question from "how big should each tool be?" to "which subset of tools should the model see right now?" The answer to the second question depends on session context, user intent, and workflow state — signals that live in the tools runtime, not in the tool definitions themselves.

Granularity is where you should start. It's the cheapest lever, it teaches you which tools matter, and it gives you the eval baseline you need for everything that comes after. But if your agent has to reach hundreds of actions across an established product portfolio, granularity alone won't get you there. The architecture around the tools has to change too.