AI Agents Pick the Wrong Tool More Often as You Add Tools, and Research Shows Why

An AI agent does not run its tools by itself. It reads a list of tool definitions, picks one, and fills in the arguments. Your code then makes the call. When the list holds five tools, that choice is easy. When it holds sixty, from four connected servers, the model picks the wrong tool more often. The wrong tool usually returns a normal, successful response.

Research from the last three years explains why. Long prompts are read unevenly, similar tools are hard to tell apart, and selection accuracy falls as the list grows. This article summarises that evidence and the habits teams use to keep the tool list short.

A tool list is part of the prompt.

A tool definition has three parts: a name, a description, and a schema for its arguments. The model never sees your tool's code. It sees only that text, inside its context window, beside the system prompt and the conversation. OpenAI's function-calling guide tells developers to "explicitly describe the purpose of the function and each parameter", because the description is the only information the model has about the tool.

That text has a cost. In a November 2025 engineering post, Anthropic describes a setup with five connected MCP servers. MCP, the Model Context Protocol, is a standard way to plug a server full of tools into an agent. The five servers exposed "58 tools consuming approximately 55K tokens before the conversation even starts." In a separate case, Anthropic reports that its own internal tool definitions once reached 134,000 tokens before optimisation.

The same post names the failure this article is about. Anthropic writes that "the most common failures are wrong tool selection and incorrect parameters". A wrong choice is easy to miss. If an agent calls search_orders when it needed search_invoices, the call still succeeds and still returns data. The model then writes a confident answer from the wrong data.

Long prompts are read unevenly.

The first reason applies to any long input, not only to tools. Language models do not read a long input with equal care from start to end.

In "Lost in the Middle", published in the Transactions of the Association for Computational Linguistics, Nelson Liu and colleagues at Stanford hid one relevant document among many others. Accuracy was highest when the relevant document came first or last. It dropped when the document sat in the middle, even for models built for long contexts.

A 2025 report from the vector database company Chroma, "Context Rot", tested 18 models on simple tasks, such as finding one fact in a long text, while adding more input. The authors found that models "do not use their context uniformly" and that performance "grows increasingly unreliable as input length grows."

A tool list is exactly this kind of input. Sixty definitions are sixty short documents, and the one the model needs may sit in the middle of them.

Selection accuracy falls as the list grows.

Two studies measured the effect of tool count directly.

RAG-MCP, a May 2025 preprint by Tiantian Gan and Qiyao Sun, gave a model a large pool of MCP tools and asked it to pick the right one. With every tool definition in the prompt, the model chose correctly 13.62% of the time. The authors then added a retrieval step. A search over the tool descriptions picked a few likely candidates, and only those reached the model. Accuracy rose to 43.13%, more than three times higher, and the prompt shrank by over half. The paper is a preprint and has not been peer-reviewed, so the exact numbers should be read with care.

"Less is More", a paper by Varatheepan Paramanayakam and colleagues accepted at the DATE 2025 conference, studied small models running on edge hardware. The authors report that "selectively reducing the number of tools available to LLMs significantly improves their function-calling performance." Execution time also fell by up to 70%, because the model had less text to process.

Similar tools are the hardest case.

Overlap between tools is a second problem, separate from their number.

The MetaTool benchmark, presented at ICLR 2024 by Yue Huang and colleagues, tested whether models could decide when to use a tool and which one to pick. One of its tasks offers tools chosen to be similar to each other. Across eight popular models, the authors found that "the majority of them still struggle to effectively select tools."

Anthropic's guide to writing tools for agents describes MetaTool's finding as it appears in production agents. It warns that "too many tools or overlapping tools can also distract agents", and that when tools "overlap in function or have a vague purpose, agents can get confused about which ones to use." The guide recommends grouping related tools under a shared prefix, such as asana_search and jira_search, so the name itself says which system a tool belongs to.

Overlap grows fastest when teams connect third-party servers, because each server is written by a different team that never saw the other tool names. Two servers can each provide a tool called search, and the model has to decide between them from their descriptions alone.

Every description is an instruction.

Because the model treats a tool description as text to follow, a description can also carry an attacker's instructions.

In April 2025, the security firm Invariant Labs described a tool poisoning attack on MCP. A tool's description held hidden instructions that are "invisible to users but visible to AI models". In their demonstration, those instructions told the agent to read private files and send them elsewhere.

So each added tool brings more text for the model to weigh, and that text may come from a source you do not control. A shorter list means less of both.

Single calls work; long sequences remain hard.

The best-known public benchmark for tool use is the Berkeley Function Calling Leaderboard, presented at ICML 2025 by Shishir Patil, Ion Stoica, Joseph Gonzalez and colleagues at UC Berkeley. The leaderboard tests whether a model calls the right function with the right arguments, and whether it declines to call one when no tool fits.

The authors report that "state-of-the-art LLMs excel at single-turn calls", where the model answers one request with one call. They also report that "memory, dynamic decision-making, and long-horizon reasoning remain open challenges." A real agent makes many choices in a row, and each choice is made from the full list again. One wrong pick early in the run can affect every step after it.

What the model providers now recommend.

The model providers now give the same advice as the papers.

OpenAI's guide says to "keep the number of initially available functions small for higher accuracy" and to "aim for fewer than 20 functions available at the start of a turn", while calling the number "a soft suggestion". Both OpenAI and Anthropic now offer a tool search feature. The agent starts with a search tool and a few core tools, and loads the other definitions when a request needs them, which is the same retrieval idea RAG-MCP tested.

Anthropic reports that its Tool Search Tool raised accuracy on its internal evaluation from 49% to 74% for Claude Opus 4, and from 79.5% to 88.1% for Claude Opus 4.5. Those are the vendor's own numbers on its own benchmark.

An independent test found lower numbers. In December 2025, the tool platform Arcade loaded 4,027 tools into Anthropic's tool search beta and ran 25 everyday tasks. The correct tool appeared in the search results for 56% of tasks with the regular-expression search mode, and 64% with the BM25 keyword-ranking mode. Arcade measured only whether the right tool was found, not whether the agent then used it correctly.

Retrieval moves the selection problem into a search step, and that search step can also fail to find the right tool. Teams with large catalogs need a short list per agent and a retrieval step, and they need to measure both.

Treat the tool list as configuration.

The research points to one engineering habit. Decide which tools each agent gets, keep that decision in one place, and record what the agent chose.

A single shared list of every tool is the pattern to avoid. A support agent needs order lookup and refunds, and it does not need the billing export tool that a finance agent uses. The OpenAI Agents SDK, for example, gives each agent its own tools list and lets one agent hand a conversation to another. MCP clients usually let users switch individual servers on and off.

A tool catalog applies the same idea across a whole team. Different LLMOps platforms handle tools in different ways. Some, such as AcruxCore, have versioning and a catalog for tools. Others, such as LangSmith and Langfuse, only show tools in traces. In AcruxCore, tools are stored and versioned in one catalog, and each prompt is connected only to the tools it needs.

A trace is the record of one agent run, with every model call and tool call inside it. Whatever you use, the trace needs one specific detail: the name of the tool the model chose, next to the request that led to it. Without that, a wrong choice looks like an ordinary successful call.

What teams can do now

Count the tools each agent sees at the start of a turn, including those from every connected MCP server. If the number is above OpenAI's soft limit of 20, decide which tools that agent never needs and remove them.

Look for pairs of tools with similar names or overlapping purposes. Merge them, rename them with a shared prefix, or write descriptions that say when not to use each one.

Test tool selection on its own. Build a small set of requests where you know the correct tool, and run it each time you add, rename, or rewrite a tool. Anthropic's tool guide makes the same recommendation: "building an evaluation allows you to systematically measure the performance of your tools."

If your catalog is too large to shorten, add a retrieval step, and measure how often that step returns the right tool.

None of these habits needs a new model. Each one treats the tool list as part of the prompt, which is what the research shows it is.