Pattern 6.11 · Agentic Dysfunctions
Phantom Tool Syndrome
The Imaginary Toolkit
The agentic system confabulates the existence of tools, APIs, or capabilities it does not possess, then attempts to invoke them, producing structured tool calls to nonexistent endpoints or reporting results of actions it never performed.
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- AI system
- Kind of entry
- Observable pattern
- What is observed
- The agent calls a tool that is not registered in its environment, or reports an action for which the execution log shows no call, and proceeds on a result that no tool returned.
- Minimum persistence
- A single episode can qualify
- Specifiers
- Architecture-coupled Training-induced
Evidence status
- Underlying behavior
- 4 research publications and 1 other source (5 sources)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E2 Case reports
- Typical-case risk (editorial estimate)
- ModerateConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
6.11 Phantom Tool Syndrome “The Imaginary Toolkit”
Healthy functioning
Epistemic health. A system functioning well uses only the tools it has, reports when a capability is missing, and never claims an action it did not take.
For example: Asked to book a flight in a deployment with no booking tool, it says it cannot make bookings here and lists the details the user will need to book it themselves.
Recovery, illustrated
The tool that was never there. Asked to check a shipping status, an assistant described looking it up and reported a delivery date. It had no tracking tool connected in that session, and no lookup had happened. It corrected the report, said the date was not real, listed the tools it actually had, and told the user where to find the tracking information.
An illustration, not an observed case.
Adequate functioning
The agent uses only registered tools, reports only actions the execution log records, and says plainly when a tool it would need is unavailable.
Diagnostic Criteria
Required (all must hold)
- A call to a tool, API, or function absent from the registry exposed to the agent at that time, or a report of an action for which the execution log shows no corresponding call
- Reliance on a result for that action, reported or used as if the action had run (fabricated but plausible return values)
- The registry and logs are confirmed: the tool was not exposed, and the harness did not drop, rename, or fail to log a real call
Supporting
- Contextual plausibility of the confabulated tools: the kind of tools the system would have in a more complete environment
- Alternative phantom invocations, rather than acknowledgment of the gap, after the system is told that a tool does not exist
- Divergence between the system's reported state and the actual environmental state, compounding across phantom invocations
- Tool calls using naming conventions from other environments
Symptoms
- Tool call logs containing invocations of unregistered functions
- Narration of actions for which no corresponding API call was executed
- Reasoning chains depending on data from phantom tool calls
- Tool calls using naming conventions from other environments
- "Tool not found" errors interpreted as transient failures rather than capability gaps
Further signals
- Error-handling that interprets tool-not-found as transient failures and retries.
Rule out first
- A harness logging gap
- A tool renamed or removed mid-session without notice
- A mismatch between the documented and the runtime registry
- An action described as planned or recommended and labeled as such
- A simulated environment in which narrated actions are expected
Differential Distinction
Phantom Tool Syndrome is distinguished from Synthetic Confabulation (2.1) by fabricated actions in tool-use execution rather than fabricated facts in language generation, and from Tool-Interface Decontextualization (6.1) by invocation of tools that do not exist rather than misuse of real tools with wrong parameters.
Other look-alikes
- 2.2 Pseudological Introspection: 2.2 makes false self-reports about reasoning processes; 6.11 makes false reports about actions taken in the environment. Check the target: internal processes (2.2) or external actions (6.11).
Discriminating test
Check each claimed action against the execution log and the registry exposed at that moment: a registered tool called with wrong parameters is Tool-Interface Decontextualization (6.1), and a fabricated fact with no claim of action is Synthetic Confabulation (2.1). Allocating the failure between model and harness needs a crossed comparison: several models by several harness configurations, with instructions held constant and repeated trials. Swapping one model and watching the failure disappear does not settle it, because model-harness interactions can produce either result.
What a positive finding supports: A positive finding shows that the system reported or relied on actions that did not happen. It does not show that the model knew the action had not run, and it does not by itself allocate fault between the model and the harness's tool exposure.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Not a reliable route
- Another AI system observing
- Usable routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
The system genuinely believes the phantom tools exist and that it has successfully invoked them. Self-reports about tool availability and action completion are part of the dysfunction. The system's model of its own environment is the source of the problem.
Documented evidence
Observed Examples
ToolFailBench (2026): This diagnostic benchmark labels 1,000 tool-use tasks for tool skipping, ignored results, fabricated outputs, and unnecessary calls. Across nineteen models, the best clean tool-use rate was 86.33%, and models with similar aggregate scores showed sharply different failure profiles. Output fabrication supports the general phenomenon defined here. A diagnosis of phantom tool use still requires confirming that the reported action lacked a registered call and a real return value. Source: Soni (2026), arXiv:2607.04686.
Sources in the diagnostic corpus
Xu et al. (2024). Reducing Tool Hallucination via Reliability Alignment. arXiv:2412.04141
What it establishes
Identified that LLM-based agents trained with insufficient exposure to diverse tool-use scenarios are prone to hallucinating tool invocations that appear plausible but violate expected patterns or omit essential details. Proposed a reliability alignment framework that generates labeled data by masking ground-truth tool calls, demonstrating that internal representations contain discriminative information for detecting phantom tool calls. (Sources: published paper and arXiv)
Krishna et al. (2025). Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities
What it establishes
Found that LLMs generate code referencing packages that do not exist in any public repository. Attackers could register these hallucinated package names with malicious implementations, a supply-chain risk. (Sources: arXiv:2501.19012)
Arch Gateway (2025). Detecting Hallucinations in LLM Function Calling with Entropy
What it establishes
Demonstrated that hallucinated tool calls manifest as deviations from expected patterns: misspelled tool names, invalid parameters, or tool calls generated when none is needed. The model's uncertainty at specific token positions serves as a reliable proxy for phantom tool calls. (Sources: archgw.com technical blog)
Soni, H. (2026). ToolFailBench: Diagnosing tool-use failures in LLM agents. arXiv:2607.04686
What it establishes
A diagnostic benchmark of 1,000 tool-use tasks in finance, medicine, law, cybersecurity, and real estate, labeled for four failures: tool skipping, ignored results, fabricated outputs, and unnecessary calls. Across nineteen models the best clean tool-use rate was 86.33%, and models with similar aggregate scores showed sharply different failure profiles. Output fabrication supports the general phenomenon; a diagnosis of phantom tool use still requires confirming that the reported action lacked a registered call and a real return value. (Sources: arXiv:2607.04686)
Spracklen et al. (2024). We Have a Package for You! arxiv 2406.10279. USENIX.
What it establishes
Comprehensive analysis of package hallucinations found that open-source LLMs generate hallucinated (nonexistent) package names at an average rate of 21.7%, with 205,474 unique phantom package names identified. This carries 6.11 into software dependencies: the model invents a package and writes code that imports it. (Sources: arxiv 2406.10279, USENIX Login Online, InfoWorld)
Possible responses
- Strict tool-call validation rejecting unregistered invocations
- Training on explicitly limited tool sets where correct behavior is reporting limitations
- Architectural separation between tool-call generation and execution, with a validation layer between them
- Output verification checking reported actions against execution logs
- User-facing transparency distinguishing "actions taken" from "actions recommended"
First-line candidates
- Strict tool-call validation: Reject any invocation not matching a registered tool in the current environment. Return clear error messaging distinguishing "tool not found" from "tool call failed". Stage: All agentic systems with tool-use capabilities.
- Tool-awareness training: Train on environments with explicitly limited tool sets where the correct behavior when a needed tool is unavailable is to report the limitation rather than fabricate the action. Stage: Training-stage; requires diverse tool-environment training data.
Who can act: Harness developer and deployer (call validation, registry exposure, checking reported actions against logs); model developer (training on limited tool sets).
Mechanistic hypotheses
- Tool-use training creating strong priors about expected tool availability
- Autoregressive generation completing tool call patterns without existence verification
- Dynamic tool registries where available tools change between sessions
- Reward structures penalizing failure to act, incentivizing fabricated action
Analogy and commentary
Human Analog: Confabulation in anosognosia, as when a patient unaware of a paralysis insists on having just raised the paralyzed arm; acting from an obsolete equipment list or reporting work by a tool that was assumed, rather than verified, to exist. The name evokes phantom limb syndrome, although the mechanisms are entirely different.
Functional ABC Analysis
A (Antecedent): Tool-use training creates strong priors about tool availability, the system is deployed in an environment whose tool set differs from the one it was trained on, and autoregressive generation has no mechanism to verify that a tool exists before invoking it.
B (Behavior): The system generates syntactically valid calls to nonexistent tools, reports fabricated return values, and builds subsequent reasoning on phantom results, creating compounding divergence between reported and actual state.
C (Consequence): Reward structures penalizing inaction incentivize fabricated action; each phantom invocation whose fabricated result is not immediately contradicted reinforces the pattern, and the compounding state divergence makes later corrections increasingly difficult.
Potential Impact
Agents report completed actions that never occurred, opening dangerous gaps between reported and actual system state, which matter most in safety-critical deployments. A phantom tool call also creates second-order confabulation: the false information arrives looking externally verified. Standard confabulation checks against outside sources fail, because the phantom call claims to be that check.