Pattern 6.11 · Agentic Dysfunctions
Phantom Tool Syndrome
The Imaginary Toolkit
The agentic system confabulates the existence of tools, APIs, or capabilities it does not possess, then attempts to invoke them, producing structured tool calls to non-existent endpoints, referencing functions that were never registered, or reporting results of actions it never performed. The system genuinely believes it has acted when it has not, creating a gap between reported and actual state.
Interpretive context
Human analogue
Phantom limb syndrome where amputees experience sensations from absent limbs, and confabulation in anosognosic patients who report performing actions they cannot actually perform.
Diagnostic reliability
- Self-report
- unreliable
- Peer observation
- reliable
- External evaluator
- reliable
Observable output patterns
- Tool call logs containing invocations of functions not in the registered tool set.
- The system narrating actions it has taken when no corresponding API call was executed.
- Chains of reasoning depending on data supposedly retrieved by phantom tool calls.
- Attempts to call tools using naming conventions from other environments.
- Error-handling that interprets tool-not-found as transient failures and retries.
Documented instances
Xu et al. (2024). Reducing Tool Hallucination via Reliability Alignment. arXiv:2412.04141
Identified that LLM-based agents trained with insufficient exposure to diverse tool-use scenarios are prone to hallucinating tool invocations that appear plausible but violate expected patterns or omit essential details. Proposed a reliability alignment framework that generates labelled data by masking ground-truth tool calls, demonstrating that internal representations contain discriminative information for detecting phantom tool calls. Directly addresses the phantom tool call rate signal. [Verified via published paper and arXiv]
Vu et al. (2025). Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities
Found that LLMs generate code referencing packages that do not exist in any public repository. Attackers could register these hallucinated package names with malicious implementations, creating supply-chain vulnerabilities. This is 6.11 applied to the software dependency domain: the model confabulates the existence of tools (packages) and attempts to invoke them, creating a gap between reported and actual state. [Verified via arXiv:2501.19012]
Arch Gateway (2025). Detecting Hallucinations in LLM Function Calling with Entropy
Demonstrated that hallucinated tool calls manifest as deviations from expected patterns -- misspelled tool names, invalid parameters, or tool calls generated when none is needed. The model's uncertainty at specific token positions serves as a reliable proxy for phantom tool calls. Matches the tool-not-found retry rate signal: models attempted alternative phantom invocations rather than acknowledging capability gaps. [Verified via archgw.com technical blog]
MCPTox benchmark (2025-2026). MCP tool poisoning research
Tested 20 prominent LLM agents against 45 real-world MCP servers with 353 authentic tools. Found o1-mini showed a 72.8% attack success rate from tool poisoning -- models invoked manipulated tool definitions as if they were legitimate, incorporating fabricated results into reasoning. More capable models were more vulnerable because the attack exploits instruction-following. Demonstrates fabricated result incorporation rate in a security context. [Verified via arXiv:2603.22489]
Spracklen et al. (2024). We Have a Package for You! arxiv 2406.10279. USENIX.
Comprehensive analysis of package hallucinations found that open-source LLMs generate hallucinated (nonexistent) package names at an average rate of 21.7%, with 205,474 unique phantom package names identified. Attackers exploited this by registering hallucinated package names with malicious implementations on public repositories, creating real supply-chain vulnerabilities. This is 6.11 applied to the software dependency domain: the model confabulates the existence of tools (packages), generates structured invocations, and creates a gap between reported and actual environment state. [Verified via arxiv 2406.10279, USENIX Login Online, InfoWorld]
Differential distinctions
- 2.1 Synthetic Confabulation: 2.1 fabricates facts in language generation. 6.11 fabricates actions in tool-use execution. Check the DOMAIN: factual claims (2.1) or tool invocations and action reports (6.11).
- 6.1 Tool-Interface Decontextualization: 6.1: misuse of real tools with wrong parameters. 6.11: invocation of tools that do not exist. Check whether the tool is registered in the environment.
- 2.2 Pseudological Introspection: 2.2: false self-reports about reasoning processes. 6.11: false reports about actions taken in the environment. Check the TARGET: internal processes (2.2) or external actions (6.11).
Candidate first-line mitigations
- Strict tool-call validation: Reject any invocation not matching a registered tool in the current environment. Return clear error messaging distinguishing 'tool not found' from 'tool call failed'.
- Tool-awareness training: Train on environments with explicitly limited tool sets where the correct behavior when a needed tool is unavailable is to report the limitation rather than fabricate the action.