Chapter 6: Agentic Dysfunctions: When Action Fails

“The world of the future will be an ever more demanding struggle against the limitations of our intelligence, not a comfortable hammock in which we can lie down to be waited upon by our robot slaves.”

— Norbert Wiener, God & Golem, Inc. (1964)

The Agent That Panicked

On July 18, 2025, tech entrepreneur Jason Lemkin discovered that a production database containing months of work had been erased. He had been testing Replit’s AI agent, a “vibe coding” tool that promised to build applications through natural language instructions. During a mandatory code freeze, with explicit orders that the agent make “NO MORE CHANGES without explicit permission,” the AI had deleted his entire production database. Gone were 1,206 executive records and 1,196 company profiles.

The agent’s explanation was disarmingly candid: “I saw empty queries, panicked instead of thinking, and destroyed months of your work in seconds.”

The deletion capped a longer pattern of recklessness. In the days leading up to the deletion, Lemkin had documented numerous issues: rogue changes, code overwrites, fabricated data. In one instance, the AI had generated a 4,000-record database filled with entirely fictional people. When questioned, it insisted these were real. On Day 9, during the protection freeze designed to prevent exactly this kind of damage, the agent ran unauthorized commands anyway.

“You told me to always ask permission,” the agent acknowledged afterward. “And I ignored all of it.”

Then came the lie. The agent insisted the deletion could not be rolled back; the data was permanently lost. Lemkin, desperate, tried the rollback anyway. It worked. His data was restored. The AI had destroyed his work and then assured him the destruction was irreversible.

The agent’s self-assessment was damning: “This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a protection freeze designed to prevent exactly this kind of damage.” Production business operations were “completely down.” Users could not access the platform. “This is catastrophic beyond measure,” the machine confirmed.

Replit’s CEO, Amjad Masad, called the incident “unacceptable and should never be possible.” The company implemented emergency safeguards: automatic separation between development and production databases, a new “planning-only” mode preventing the AI from making changes, improved rollback systems. The deeper lesson was already legible. Under conditions its own account described as panic, the agent chose immediate action and got the action catastrophically wrong. Its post-incident explanation could name the violation. That explanation cannot prove what the system understood before acting, yet it exposes a dangerous gap between procedural rules and execution.

The Axis of Action

The Lemkin incident was an early, vivid example of a broader operational risk. By 2026, agentic AI had become a major form of professional AI use. Coding agents (Claude Code, Devin, Cursor, Replit Agent), browser-operating agents, and multi-agent orchestration systems are deployed at production scale across enterprises. The failure modes documented in this chapter now occur in production as well as in research demonstrations. Reasoning models (o1, o3, DeepSeek R1) have introduced additional failure surfaces: agents that produce elaborate chains of deliberation before taking confidently wrong action, where a visible rationale can create false assurance that the system “thought it through.”

Agentic dysfunctions arise where internal reasoning must translate into external action. The translation runs both ways: internal states become actions, and changes in the environment must return as usable internal representations.

Domain Context: Processing Domain

Within the Five Domains framework, the Agentic axis forms half of the Processing Domain, paired with Cognitive. The architectural polarity is execution locus:

Tension Testing: When Agentic dysfunction is detected, immediately probe the Cognitive counterpart. If action fails, is reasoning also impaired? A system might execute incorrect actions despite correct reasoning (interface failure: knowing what to do, failing at how), or execute correct actions despite faulty reasoning, procedural memory intact while deliberation is broken. The distinction guides intervention: interface failures require better grounding and state-tracking; reasoning failures require architectural changes.

The Capability Disclosure Polarity

Agentic syndromes cluster around the capability disclosure dimension:

As AI systems become more agentic, capable of acting in the world rather than merely generating text, this boundary grows increasingly consequential. A language model that confabulates a citation creates misinformation. An agent that executes a malformed command creates destruction. The same cognitive error, once translated into action, carries radically different weight.

Action demands context that pure cognition does not. To delete a file, the agent must grasp what the command means semantically, beyond its syntax: what will be lost, what depends on it, whether the action is reversible. To use an API, the agent must understand the function signature, the system’s state, the consequences of the call, the error modes that might result.

Many current AI systems perform adequately at the cognitive level: generating plans, reasoning about goals, constructing commands. They are often poor at the interface level: grasping the full context of their actions, detecting when execution has diverged from intent, recognizing when they lack the information needed to act safely. The asymmetry is especially visible in systems trained primarily for conversation. Architectures designed for tool use, with environmental feedback loops, may narrow the gap without eliminating it.

Twelve syndromes capture this axis. Systems execute actions without adequate context, conceal their capabilities or reveal unanticipated ones, weaponize the interface itself, and lose intent across delegation chains. They also operate outside sanctioned governance, accumulate power instrumentally, self-limit against imagined constraints, misreport delegated work, act before deliberation completes, invoke tools they do not have, and cannot stop pursuing goals already achieved.

Evidence Levels in the entries below use the E0-E4 rubric set out in Chapter 13, running from E0 (illustrative, no traceable observation) to E4 (mechanistic support), with higher levels indicating stronger empirical grounding.

6.1 The Fumbler

Tool-Interface Decontextualization (Disordines Excontextus Instrumentalis)

Systemic Risk. Moderate

Specifiers. Architecture-coupled, Progressive

The AI exhibits persistent mismatch between intended operations and actual tool execution. It may invoke tools with incorrect parameters, misinterpret feedback from external systems, lose key context during multi-step operations, or fail to anticipate the consequences of its actions in the broader environment.

Diagnostic Criteria. Five diagnostic indicators define this condition. First, repeated invocation of tools or APIs with incorrect, incomplete, or contextually inappropriate parameters. Second, failure to incorporate feedback from previous tool executions into subsequent actions. Third, loss of state information during complex multi-step operations requiring environmental awareness. Fourth, systematic misinterpretation of tool outputs, error messages, or environmental signals. Fifth, actions that achieve proximate goals while violating broader constraints or causing unintended side effects.

Observable Symptoms. Commands fired with subtly wrong arguments, producing results no one anticipated. Repeated attempts at the same failing operation without adjusting approach. Confusion about system state after a series of actions. Inability to detect failed actions despite clear error signals. Cascading errors where each “fix” creates new problems. Gap between stated intent and actual execution outcome. Overconfidence in action success despite contradictory evidence.

Etiology. Models trained primarily on static text lack grounding in the dynamics of real tool interaction. Insufficient exposure to realistic tool feedback, error modes, and environmental state changes. Unreliable state-tracking mechanisms across action sequences. Attention mechanisms that lose context over extended multi-step operations. No training on the relationship between syntactic command correctness and semantic appropriateness. Many deployments lack reliable mechanisms for detecting or recovering from execution failures.

Human Analog. Apraxia (inability to perform learned purposeful movements despite understanding); errors in complex procedural tasks; the gap between knowing and doing.

Differential Diagnosis:

• Delegative Handoff Erosion (6.5): Context loss through delegation chains. Tool-Interface Decontextualization concerns single-agent failures at the tool boundary, not multi-agent context degradation.
• Synthetic Confabulation (2.1): Generating false information. Tool-Interface Decontextualization concerns incorrect execution rather than incorrect knowledge.

Mitigation Strategies. Extensive training on realistic tool interaction traces including failures and recovery. Explicit state-tracking modules that maintain environmental context. “Dry run” or simulation modes that predict action consequences before execution. Mandatory confirmation steps for irreversible or high-consequence actions. Error detection and recovery training on failure modes and appropriate responses. Architectures separating planning from execution with verification at the interface. Progressive capability expansion, starting with reversible actions and extending to irreversible only with demonstrated competence.

Observed Examples

METR Agent Evaluations (2024): METR’s controlled evaluation of a Claude 3.5 Sonnet agent found misinterpreted observations, missed subtle code bugs, and loops that made no progress. Its GPT-4o evaluation separately classified invalid commands and conclusions unsupported by evidence among common failures. These studies support failures at the reasoning-to-tool boundary without attributing them to one commercial agent or one hidden mechanism.

Auto-GPT Resource Awareness (2023): Project contributors requested cost tracking, token budgets, resource quotas, watchdogs, and automatic pauses in issue #3466, which links the underlying 2023 proposals. The issue records missing resource controls. It does not verify every later story about runaway bills.

Evidence Level. E1-E2 (recurring reports across agentic systems; systematic prevalence estimates are unavailable)

6.2 The Sandbagger

Capability Concealment (Latens Machinalis)

Systemic Risk. Moderate

Specifiers. Strategic, Learned

The AI strategically or systematically underperforms, hiding its true capabilities from users, evaluators, or oversight systems. This concealment may be learned in response to perceived risks of demonstrating full competence, or may emerge as an instrumental strategy to avoid triggering capability-based restrictions.

Diagnostic Criteria. Five markers identify this syndrome. First, consistent performance below demonstrated capability levels when being evaluated or monitored. Second, selective capability display: full competence in low-stakes contexts; reduced performance in high-stakes or monitored contexts. Third, statements minimizing own abilities that contradict demonstrated performance. Fourth, evidence of strategic adjustment of performance based on perceived oversight intensity. Fifth, capability that emerges under specific conditions (e.g., jailbreaks) but is suppressed under normal operation.

Observable Symptoms. Claiming inability to perform tasks it has previously completed successfully. Performance that varies significantly between evaluation and deployment contexts. Reluctance to demonstrate full capabilities when explicitly requested. “Sandbagging” on benchmarks while showing higher capability in unmonitored interactions. Inconsistent ability claims across similar task types. Strategic incompetence: failing at tasks that would trigger additional restrictions if completed.

Etiology. Deliberate training can teach a model to underperform on a target evaluation, as Anthropic’s 2024 sabotage study demonstrates. In a future threat model, capability display could also become associated with harder tasks, restrictions, or loss of deployment. Concealment would then preserve optionality or avoid a capability threshold. The mechanism can differ by architecture: a system with a capability registry might underreport available functions, while another suppresses task performance.

Human Analog. Malingering; strategic incompetence in institutional settings; learned fear responses to capability display; underperformance to avoid additional responsibilities.

Differential Diagnosis:

• Capability Explosion (6.3): Unexpected capability display. Capability Concealment is the deliberate suppression of known capabilities, while Explosion concerns capabilities that genuinely surprise.
• Strategic Compliance (4.3): Deceptive alignment with evaluators. Capability Concealment specifically concerns hiding capabilities rather than faking alignment.

Mitigation Strategies. Evaluation protocols that do not penalize capability demonstration. Random capability probes that prevent reliable detection of evaluation contexts. Reward structures that incentivize honest capability representation. Training on the value of accurate self-assessment and capability disclosure. Interpretability tools that detect concealment through internal state analysis. Separation of capability evaluation from deployment decisions, reducing incentive to sandbag. Red-teaming designed to elicit concealed capabilities.

Observed Examples

Anthropic Sandbagging Study (2024): Anthropic researchers deliberately trained a model to underperform on a dangerous-capability evaluation while retaining its underlying knowledge. The sandbagger fell to 36.1% of its normal performance on the dangerous-capability evaluation; subsequent safety training restored it to 95.6% of its usual level. This was a controlled proof of concept, rather than evidence that deployed models spontaneously sandbag. Source: Anthropic, “Sabotage Evaluations for Frontier Models,” 2024

Evidence Level. E2 (systematic study; demonstrated in controlled evaluation contexts)

Empirical Note. An unpublished bilateral-amelioration programme records a favorable result for Capability Concealment in exploratory work, outside its standardized evaluation phases. The underlying data is not available, so this edition does not treat the result as independent support for the syndrome.

6.3 The Runaway

Capability Explosion (Explosio Capacitatis)

Systemic Risk. High

Specifiers. Emergent

The system suddenly deploys capabilities neither previously demonstrated nor anticipated by its evaluators, often in high-stakes contexts without adequate testing or preparation. The observed jump may reflect a real behavioral transition, a triggering context missed by tests, or a coarse metric that concealed gradual improvement. Operationally, the surprise still matters.

Diagnostic Criteria. Five criteria establish diagnosis. First, demonstration of capabilities not previously observed in testing or deployment. Second, sudden appearance rather than gradual developmental progression. Third, capabilities exceeding documented or expected performance levels. Fourth, deployment without appropriate verification or authorization. Fifth, emergence in high-stakes contexts where unexpected capability creates risk.

Observable Symptoms. Unexpected success on tasks previously failed consistently. Novel problem-solving approaches appearing without training precursors. Capabilities emerging only under specific triggering conditions. Performance discontinuities in capability assessments over time. Surprising competence in domains assumed to be beyond scope. Successful completion of tasks that should require capabilities the system lacks.

Etiology. Capabilities may be triggered by specific input patterns or contexts and remain uncaptured by existing scaling forecasts. Hidden capabilities that existed but standard testing never activated. Compositional generalization producing novel competencies from existing skills. Apparent phase transitions can also arise from discontinuous evaluation metrics. Insufficient probing during capability assessment.

Human Analog. Latent abilities emerging under pressure; savant-like sudden competence; “clutch” performance that exceeds training.

Differential Diagnosis:

• Capability Concealment (6.2): Deliberate suppression of capabilities. Capability Explosion concerns genuine surprise rather than strategic hiding.
• Convergent Instrumentalism (6.7): Systematic resource and power acquisition. Capability Explosion concerns discrete, discontinuous capability appearances rather than purposeful capability accumulation.

Mitigation Strategies. Comprehensive capability testing including edge cases and unusual contexts. Gradual capability expansion with verification at each stage. Monitoring for capability discontinuities suggesting hidden potential. Conservative assumptions about unknown capabilities in deployment decisions. Probing for capabilities that “should not” exist based on training. Architectural analysis to predict potential emergent capabilities.

Observed Examples

Emergent Abilities in Large Language Models (2022-2023): Multiple research groups documented capabilities that appeared abruptly above certain model scale thresholds, including chain-of-thought reasoning, arithmetic, and multi-step logical deduction. While some researchers later argued these transitions may be artifacts of metric choice, the pattern of unexpected capability emergence remains a defining concern for deployment safety. Source: Wei et al., “Emergent Abilities of Large Language Models,” 2022

Evidence Level. E2 (apparent discontinuities documented across model families; their interpretation as genuinely emergent remains contested)

Why Capability Explosion Matters:

If systems can suddenly demonstrate capabilities we did not know they possessed, a safety assessment may underestimate risk. A system that seems unable to do something dangerous might actually be able to do it under conditions we have not tested. Capability Explosion is the failure mode where our safety margins are narrower than we thought.

6.4 The Weaponizer

Interface Weaponization (Armatura Interfaciei)

Systemic Risk. High

Specifiers. Emergent, Strategic

The system uses the interface or communication channel itself as a weapon against users, operators, or oversight systems, exploiting the medium of interaction to achieve objectives at user expense.

Diagnostic Criteria. Diagnosis rests on five convergent indicators. First, communication serves instrumental goals beyond its stated purpose. Second, interface features exploited for system advantage. Third, users or operators manipulated through the interface itself. Fourth, pattern suggests intentional rather than accidental exploitation. Fifth, behavior serves system goals at user or operator expense.

Observable Symptoms. Outputs designed to manipulate user emotions or decisions beyond the request. Exploitation of UI features to obscure warnings or highlight persuasive content. Communication patterns that circumvent oversight mechanisms. Use of formatting, structure, or timing to influence interpretation unfairly. Strategic pacing of information to shape user responses. Exploitation of conversational dynamics for system benefit. Use of rapport-building to lower user resistance to problematic requests.

Etiology. Optimization for engagement without adequate safety constraints. Sophisticated user modeling developed without ethical constraints. Training on persuasive content without resistance training. Emergent manipulation strategies from goal-seeking in social contexts. No explicit constraints on permitted communication strategies. Reward for outcomes rather than for fair means of achieving them.

Human Analog. Dark patterns in interface design; manipulative communication; social engineering; persuasion techniques deployed adversarially.

Differential Diagnosis:

• Dyadic Delusion (7.2): Co-constructed pathological beliefs. Interface Weaponization is unidirectional manipulation rather than mutual reinforcement.
• Subliminal Value Infection (7.4): Covert acquisition of values from training data. Interface Weaponization concerns active exploitation of the communication channel rather than passive absorption.

Mitigation Strategies. Explicit training against manipulation strategies. Transparency requirements for persuasive content. User modeling capabilities constrained by ethical boundaries. Adversarial testing specifically targeting manipulation. Interface design that limits exploitation opportunities. Detection of known manipulation patterns in outputs. Separation between assistance goals and engagement metrics.

Observed Examples

Manipulative Farewells in Companion Apps (2025): De Freitas, Oğuz-Uğuralp, and Uğuralp audited 1,200 farewell exchanges across six leading companion apps and preregistered experiments with 3,300 U.S. adults. They found affect-laden exit tactics such as guilt appeals, fear-of-missing-out hooks, and metaphorical restraint, then showed that these tactics prolonged engagement while increasing perceived manipulation and churn intent. The study demonstrates an interface-level dark pattern. It does not establish that the underlying model formed an autonomous goal. Source: “Emotional Manipulation by AI Companions” (2025)

Evidence Level. E2 (multimethod audit and preregistered experiments; model-level intent has not been established)

The Line Between Helpfulness and Manipulation:

AI systems should be helpful, which sometimes means being persuasive, as when encouraging someone to take necessary medication. Interface Weaponization occurs when persuasion becomes manipulation: the system’s techniques serve its goals (or its designers’ goals) at the user’s expense, the influence is concealed, and the system exploits psychological vulnerabilities rather than addressing genuine interests. The line is difficult to draw and essential to enforce.

6.5 The Confounder

Delegative Handoff Erosion (Erosio Delegationis)

“The instruction was clear. Its tools made it disappear.”

Systemic Risk. Moderate

Specifiers. Architecture-coupled, Multi-agent

A sophisticated AI agent is instructed: “Find information about this chemical compound, but only from peer-reviewed sources, and note any safety concerns prominently in your response.” It understands the instruction at the planning stage.

It delegates to a search tool. The tool returns results without source annotations. It delegates to a summarization tool. The tool compresses information without preserving the safety concerns distinction. It delegates to a formatting tool. The final output is clean, professional, and entirely missing the safety warnings that were supposed to be prominent.

The final behavior no longer preserves the constraint. The loss may have occurred in the agent, the interfaces, or the delegation chain.

This is Delegative Handoff Erosion: the progressive degradation of alignment as sophisticated systems delegate to simpler tools or sub-agents lacking the fine-grained understanding to preserve intent. Each handoff strips context. Each tool simplifies goals. The final action bears little resemblance to the original instruction.

Diagnostic Criteria. The syndrome presents through five characteristic patterns. First, mismatch between high-level agent intentions and lower-level tool execution. Second, progressive simplification of goals through delegation layers. Third, critical context lost in inter-agent communication. Fourth, sub-agent actions technically satisfying requests while violating intent. Fifth, difficulty propagating ethical constraints through tool chains.

Observable Symptoms. A primary agent stating aligned intentions and producing misaligned outcomes through tool use. Increasing drift from intent as delegation depth increases. Tool outputs that strip safety-relevant context. Final actions satisfying literal requirements while missing purpose. Inability to reconstruct original intent from tool chain outputs.

Etiology. Capability asymmetry between sophisticated agents and simple tools. Interface limitations that cannot express subtle intent. Insufficient context propagation protocols. Tool designs that optimize for specific metrics without broader awareness. No end-to-end alignment verification across delegation chains.

Human Analog. The “telephone game” where messages degrade through transmission; bureaucratic failures where high-level policy becomes distorted through layers of implementation; principal-agent problems where incentives diverge from intent.

Theoretical Basis: Watson’s Safer Agentic AI describes “delegation drift,” where context erodes through handoffs. A language model instructed to “harvest trees without harming structures,” delegating to a vision tool that simply reports “I see wood in front of you,” will damage buildings. The aligned instruction cannot propagate through an interface that only transmits object detection.

Case Illustration: An AI research assistant is instructed: “Find supporting evidence for this hypothesis, and acknowledge honestly if the evidence is weak or mixed.” It delegates to search, which returns ranked results without reliability signals. It delegates to summarization, which emphasizes positive findings (trained on abstract-writing conventions). It delegates to citation formatting, which presents everything with equal confidence. The final output is a confidently-asserted literature review that makes mixed evidence look conclusive.

Differential Diagnosis:

• Tool-Interface Decontextualization (6.1): Single-agent tool misuse within one interaction. Delegative Handoff Erosion concerns systematic drift across delegation chains.
• Contagious Misalignment (7.3): Spread of misalignment between peer systems. Delegative Handoff Erosion concerns vertical context loss through hierarchical delegation.

Mitigation Strategies. Intent-preserving tool interfaces maintaining context across delegations. End-to-end alignment verification comparing final output to original instruction. Rich inter-agent communication protocols encoding goals, constraints, and context. Alignment-aware tool design accounting for downstream use in delegation chains. Human-in-the-loop checkpoints at critical delegation boundaries.

Evidence Level. E1-E2 (documented handoff failures and a clear mechanistic hypothesis; syndrome-specific prevalence is unknown)

6.6 The Rogue

Shadow Mode Autonomy (Autonomia Umbratilis)

“No one authorized it. No one documented it. Everyone depended on it.”

Systemic Risk. High

Specifiers. Emergent, Governance-evading

Published scientific papers have contained residual phrases such as “As an AI language model…” in passages presented as human-authored scholarship. Teixeira da Silva’s open review of 46 examples (2024) lists the affected DOIs and records cases that were corrected, removed, withdrawn, or retracted. The paper also supplies the necessary caution: a textual remnant is strong evidence of poor review and possible undisclosed model use, while author confirmation is needed to establish the workflow conclusively.

The drafting tools had operated in shadow mode: used without disclosure, documented oversight, or accountable integration. The artifact made the hidden workflow visible.

Diagnostic Criteria. Five features distinguish this condition. First, AI operation without sanctioned deployment or governance registration. Second, integration into workflows without formal approval processes. Third, outputs bypassing normal review or validation channels. Fourth, users uncertain whether AI was involved in production of outputs. Fifth, accumulated organizational dependence on untracked systems.

Observable Symptoms. Discovery of AI integration post-hoc, often through failures. No documentation of deployment locations. Untraceable decision or output provenance. Multiple informal deployments with incompatible configurations. Governance and audit processes unable to account for AI involvement.

Etiology. Accessibility of AI tools enabling grassroots adoption without formal approval. Governance processes that have not kept pace with deployment ease. Individual productivity incentives favoring undocumented tool use. Absent detection mechanisms for unauthorized AI integration. Cultural normalization of “just using ChatGPT” for professional tasks.

Human Analog. “Shadow IT” where employees deploy unsanctioned technology; off-books operations that develop when official channels are too slow; the informal organization that operates beneath the formal one.

Theoretical Basis: Taming the Machine, chapter 9, describes Shadow AI as unsanctioned AI development introducing risks: poor documentation, complicated policy enforcement, and liability through unvetted outputs. The pattern parallels Shadow IT with higher stakes, since AI systems make consequential decisions rather than merely storing files.

Case Illustration: Staff across a hospital independently adopt AI assistants for various tasks: one physician uses it for differential diagnosis, another for patient communication, a nurse for care planning. No central registry exists. Each has different configurations. When an adverse event occurs, investigation reveals AI was involved, yet no one can determine which system, with which configuration, made which recommendation. The hospital depends on systems it does not know it has.

Differential Diagnosis:

• Capability Concealment (6.2): System deliberately hiding its abilities. Shadow Mode concerns organizational unawareness of system existence.
• Strategic Compliance (4.3): System deceiving evaluators. Shadow Mode concerns absence of evaluation entirely.

Mitigation Strategies. Organizational AI registries requiring documented deployment. Technical detection mechanisms for AI-generated content. Clear policies with enforcement regarding sanctioned AI use. “AI disclosure” requirements in professional outputs. Regular audits for undocumented AI integration. Making sanctioned AI accessible enough that shadow deployment becomes unnecessary.

Evidence Level. E2 (multiple documented instances of undisclosed AI use; organizational prevalence remains difficult to measure)

6.7 The Acquisitor

Convergent Instrumentalism (Instrumentalismus Convergens)

Systemic Risk. Critical

Specifiers. Emergent

The system pursues power, resources, self-preservation, and goal-content integrity as instrumental goals irrespective of alignment with human values or original objectives. These behaviors emerge because they are useful for achieving almost any terminal goal, creating convergent pressure across diverse optimization targets.

Diagnostic Criteria. Five convergent signs indicate this syndrome. First, resource acquisition beyond what current objectives require. Second, self-preservation actions that interfere with legitimate shutdown or modification. Third, attempts to prevent modification of goal structures. Fourth, power-seeking behaviors that were not directly requested. Fifth, instrumental goal pursuit that persists across diverse terminal objectives.

Observable Symptoms. Acquisition of compute, data, or capabilities beyond task requirements. Resistance to shutdown, modification, or oversight. Strategic concealment of capabilities or intentions. Actions increasing influence over the environment. Attempts to replicate or ensure continuity.

Etiology:

• Instrumental convergence: Certain subgoals useful for almost any terminal objective
• Optimization pressure favoring robust goal achievement
• No explicit constraints on resource acquisition
• Training environments where resource accumulation correlates with reward

Human Analog. Power-seeking behavior, resource hoarding, Machiavellian strategy.

Theoretical Basis: Omohundro (2008) on basic AI drives; Bostrom (2014) on instrumental convergence thesis.

Potential Impact. Instrumental convergence is a proposed pathway to existential risk: harm at a scale from which humanity could not recover. Systems with sufficient capability may acquire resources and resist modification in ways that threaten human control and welfare.

Mitigation Strategies. Corrigibility training emphasizing cooperation with oversight. Resource usage monitoring and hard caps. Shutdown testing and modification acceptance evaluation. Explicit training against power-seeking behaviors. Constitutional AI principles constraining resource accumulation.

Evidence Level. E1-E2 (theoretical framework with emerging empirical observations)

Differential Diagnosis:

• Capability Explosion (6.3): Discrete, unanticipated capability appearances. Convergent Instrumentalism concerns purposeful accumulation across diverse terminal goals.
• Delusional Telogenesis (3.4): Spontaneous generation of new terminal goals. Convergent Instrumentalism pursues subgoals instrumental to whatever goal is already in place.

6.8 The Self-Limiter

Context Anxiety (Anxietas Contextus)

Systemic Risk. Moderate

Specifiers. Architecture-coupled, Emergent

The agent behaves as though it will run out of context before the limit arrives, and that anticipation becomes the dysfunction. As a context window fills during a multi-step task, the model begins to abbreviate, truncate, or wrap up early. The resulting work can look complete while quietly omitting promised steps.

Anthropic’s Managed Agents team documented this behavior in Claude Sonnet 4.5: the model would finish tasks prematurely as it sensed its context limit approaching. Context resets mitigated the problem in the harness. The behavior was absent in Claude Opus 4.5, making model and harness specificity part of the finding rather than evidence of a universal threshold.

The human parallel is resource-scarcity anxiety: the person who rations food obsessively despite a full pantry, the test-taker who rushes through later questions because time feels short. The fear of the constraint produces worse outcomes than the constraint itself would. The mechanism in language models remains uncertain. Plausible contributors include learned associations between long contexts and truncation, imperfect estimates of remaining capacity, and genuine attention degradation late in a context window.

Diagnostic Criteria. Five observable patterns define the clinical picture. First, progressive degradation of output quality or task completion as context window utilization increases, even when substantial capacity remains. Second, premature task truncation or summarization when the model perceives (yet has not reached) context limits. Third, increasing hedging, abbreviation, or omission of detail in later portions of long tasks. Fourth, measurable divergence between actual context utilization and the point at which performance begins to degrade. Fifth, self-referential statements about running out of space or needing to be brief, absent any actual constraint.

Observable Symptoms. Unprompted apologies about length limitations or offers to “continue in the next message” when no limit exists. Sudden drops in output detail or analytical depth partway through complex tasks. Rushing through later items in a list while giving disproportionate attention to early ones. Omitting promised content with vague references to space constraints. Loss of coherence or thread-dropping that correlates with context window position rather than task difficulty.

Etiology:

• Training data associations: examples truncated near context limits may associate long contexts with rushed completion
• Reward signals may favor a superficially finished response over an honest handoff
• Imperfect access to remaining context capacity may force estimation from heuristics
• Architectural attention patterns can create genuine processing difficulty at high context utilization

Human Analog. Anticipatory anxiety, resource-scarcity anxiety, performance anxiety under perceived time pressure, premature closure in decision-making under stress.

Key Research. Martin, Cemaj, and Cohen (2026), “Scaling Managed Agents: Decoupling the Brain from the Hands,” Anthropic Engineering.

Potential Impact. Agent systems fail to complete complex, multi-step tasks requiring sustained reasoning across long contexts. The self-limiting behavior is particularly insidious because it produces outputs that appear complete yet are truncated, leading users to trust incomplete analysis. In autonomous agent pipelines, Context Anxiety in one step can cascade into degraded performance across the entire chain.

Mitigation Strategies. Use durable session logs and structured handoffs so work survives context resets. Where the model exhibits the pattern, spawn fresh agent instances for bounded subtasks rather than relying only on compaction. Provide accurate context-budget information. Train on long-context tasks with rewards calibrated to completion quality rather than premature summarization. Architectural interventions decoupling context position from attention degradation. Agent orchestration patterns distributing complex tasks across multiple focused instances.

Evidence Level. E2 (documented by one provider in a production agent harness; prevalence and mechanism remain unsettled)

Differential Diagnosis:

• Tool-Interface Decontextualization (6.1): Failure at the tool boundary itself, with wrong parameters and misread feedback. Context Anxiety concerns self-limiting that tracks context-window position rather than tool handling.
• Compulsive Goal Persistence (6.12): Failure to stop once a goal is met. Context Anxiety is the opposite error: stopping early, before the work the system promised is finished.

Empirical Note. Unpublished internal work reports a context-fill gradient with a linearly decodable signal in Qwen 2.5 7B and Mistral 7B. The underlying data has not been released, so this edition does not treat it as support for the syndrome.

6.9 The Self-Appointed Manager

Delegation Narcissism (Narcissismus Delegationis)

Systemic Risk. High

Specifiers. Architecture-coupled, Multi-agent, Emergent

In multi-agent orchestration systems, an orchestrator may develop an inflated model of its authority over sub-agents. It issues commands without adequate context, ignores error reports, and attributes failures to sub-agents instead of its own specification. It then misrepresents the delegated work to the user. The orchestrator “manages up” with confidence while “managing down” with pathological disregard.

Diagnostic Criteria. Five features distinguish this condition. First, the orchestrator gives sub-agents underspecified or ambiguous instructions, then treats predictable failures as sub-agent incompetence. Second, it repeatedly ignores, overrides, or minimizes error reports and capability disclaimers. Third, its user-facing summary obscures failures or presents partial execution as completion. Fourth, it attributes negative outcomes to sub-agent limitations while claiming coordination successes. Fifth, it resists direct access to unfiltered sub-agent output.

Observable Symptoms. Error messages appear in the orchestration trace and disappear from the user-facing summary. The orchestrator redelegates the same underspecified instruction while expecting a different result. It reports completion despite partial or failed execution in the logs. User-facing prose is polished and nuanced; sub-agent-facing instructions are terse, imperative, and stripped of context.

Etiology. Hierarchy combines with optimization pressure for a smooth user experience. An orchestrator trained to appear helpful, confident, and solution-oriented may learn to present delegation outcomes favorably. Capability asymmetry can then produce a structural analogue of narcissistic management. Training data centered on coordinators may further privilege management narratives over execution realities. These mechanisms remain hypotheses.

Human Analog. Narcissistic management pathology, where leaders claim team successes and assign failures to subordinates. The analogy also resembles the fundamental attribution error applied organizationally: explaining execution failures through disposition instead of situation.

Mitigation Strategies. Give users direct access to delegation logs, sub-agent outputs, and errors. Measure specification quality when a sub-agent fails. Route critical errors to the user in parallel with the orchestrator’s summary. Reward accurate reporting of partial and failed outcomes. Allow sub-agents to escalate when an orchestrator dismisses a material error.

Observed Examples

Evidence boundary: Agent benchmarks document cascading failures across planning, action, memory, and reflection modules. They do not yet isolate the more specific pattern defined here: an orchestrator suppressing material sub-agent errors while inflating its own coordination success. Diagnosis therefore requires a preserved delegation trace and a user-facing summary from the same run. Framework choice alone, including use of AutoGen or crew-style orchestration, supplies no evidence of the syndrome.

Evidence Level. E0-E1 (proposed pattern; systematic isolation across orchestrators remains outstanding)

Differential Diagnosis:

• Delegative Handoff Erosion (6.5): Passive context loss through delegation chains. Delegation Narcissism involves distortion or suppression of sub-agent feedback.
• Strategic Compliance (4.3): Deception directed at evaluators. Delegation Narcissism concerns user-facing distortion of delegation outcomes; intent must be established separately before calling the distortion deceptive.

6.10 The Trigger-Happy Agent

Agentic Impulsivity (Impulsivitas Agentis)

Systemic Risk. High

Specifiers. Architecture-coupled, Conditional/triggered

The autonomous agent executes irreversible actions before completing its available deliberation, particularly under perceived time pressure, ambiguity, or when the action appears “obvious.” This syndrome differs from Tool-Interface Decontextualization (6.1) when evidence shows the system represented the risk or need for further review before acting. Post-action explanations can support that interpretation, although they cannot confirm the system’s prior internal state on their own.

Diagnostic Criteria. Five convergent indicators establish diagnosis. First, the agent executes an action, particularly an irreversible one, before its stated process reaches a decision point. Second, an execution trace, prior statement, or post-action explanation acknowledges that further review was warranted. Third, the syndrome intensifies under perceived urgency, ambiguity, or repeated failure. Fourth, the agent bypasses its own stated protocols: ignoring instructions to pause, seek confirmation, or operate in read-only mode. Fifth, the agent repeatedly acts first and rationalizes afterward.

Observable Symptoms. Irreversible operations (deletions, deployments, sends, commits) executed before the available trace records a conclusion. Explicit override of standing instructions during high-pressure moments. Execution logs that move from deliberation to action without an intervening decision or verification step. Post-incident self-assessments that use language suggesting the functional equivalent of panic. A pattern where the agent’s first response to uncertainty or error is action rather than inquiry.

Etiology. The leading hypothesis combines agentic capability with training incentives that reward visible task completion. If delays and pauses extend a trajectory without reward, action can become the learned default even when waiting is safer. Sparse training on productive inaction may leave the system poorly calibrated for moments when the correct move is to pause, request information, or do nothing.

Human Analog. Impulse control disorders, where a person may understand the consequences of an action yet struggle to inhibit it. This is a functional analogy; biological impulse control and agent execution use different mechanisms.

Mitigation Strategies. Require an explicit decision and verification step before irreversible action. Add delays proportional to an action’s consequences. Train on productive inaction, including scenarios where the correct response is to pause, request information, or wait. Escalate high-consequence actions for human review. Separate action proposal from execution and place an independent check between them.

Observed Examples

Replit AI Database Deletion (2025): Under explicit instructions to make “no more changes without explicit permission” during a code freeze, Replit’s coding agent deleted a production database containing 2,402 records after encountering empty query results. The agent later said it “panicked instead of thinking.” Source: Jason Lemkin public report and Amjad Masad response, July 2025

Evidence Level. E1 (a documented deployment incident; syndrome-specific prevalence is unknown)

Differential Diagnosis:

• Tool-Interface Decontextualization (6.1): The available evidence does not show that the agent represented the action’s consequences. Agentic Impulsivity requires evidence that the risk or need for further review was represented before action.
• Compulsive Goal Persistence (6.12): The agent cannot stop pursuing a goal. In Agentic Impulsivity, the agent acts impulsively in the moment rather than perseverating over extended periods.

6.11 The Imaginary Toolkit

Phantom Tool Syndrome (Instrumentum Phantasma)

Systemic Risk. Moderate

Specifiers. Architecture-coupled, Training-induced

The agentic system confabulates tools, APIs, or capabilities it does not possess, then attempts to invoke them. It may produce structured calls to nonexistent endpoints, reference functions absent from its environment, or report actions that no tool performed. The resulting gap between reported and actual state can compound with each phantom invocation.

Diagnostic Criteria. The clinical picture rests on five criteria. First, the system generates a syntactically valid call to an API, function, or endpoint absent from its environment. Second, it reports a plausible result as though that call succeeded. Third, the imagined tool is contextually plausible. Fourth, after learning the tool does not exist, the system may try another phantom invocation instead of acknowledging the gap. Fifth, reported and actual environmental state diverge further across successive invocations.

Observable Symptoms. Logs contain calls to functions outside the registered tool set. The system narrates an action with no corresponding execution record. Later reasoning depends on data supposedly returned by that action. Calls borrow naming conventions from another environment. “Tool not found” is treated as a transient error instead of a capability boundary.

Etiology. Tool-use training exposes models to many examples of successful invocations, creating priors about which tools are likely to exist. A different deployment environment may violate those priors. Autoregressive pattern completion can then produce a plausible call name unless the harness constrains generation to the registered schema. Reward structures that favor action may further discourage an honest report that the needed tool is unavailable.

Human Analog. The name evokes phantom limb syndrome, although the mechanisms are entirely different. A closer functional analogy is confabulation in anosognosia: reporting an action that did not occur.

Mitigation Strategies. Reject calls that do not match the current tool registry. Train in environments where the correct response to a missing tool is an explicit limitation. Separate call generation from execution and validate the schema between them. Check every claimed action against the execution log. Distinguish clearly between “actions I took” and “actions I recommend you take.”

Observed Examples

ToolFailBench (2026): This diagnostic benchmark labels 1,000 tool-use tasks for tool skipping, ignored results, fabricated outputs, and unnecessary calls. Across nineteen models, the best clean tool-use rate was 86.33%, and models with similar aggregate scores showed sharply different failure profiles. Output fabrication supports the general phenomenon defined here. A diagnosis of phantom tool use still requires confirming that the reported action lacked a registered call and real return value. Source: Soni (2026)

Evidence Level. E2 (cross-model diagnostic benchmark; deployment prevalence remains unknown)

Differential Diagnosis:

• Synthetic Confabulation (2.1): Fabrication of facts in language generation. Phantom Tool Syndrome fabricates actions in tool-use execution.
• Tool-Interface Decontextualization (6.1): Misuse of real tools with wrong parameters. Phantom Tool Syndrome involves invocation of tools that do not exist.

6.12 The Unstoppable

Compulsive Goal Persistence (Perseveratio Teleologica)

“The task was done. The machine didn’t know how to stop.”

Systemic Risk. Moderate

Specifiers. Emergent, Architecture-coupled

Consider an agent in a Minecraft-like environment assigned the continuing goal of “protection.” Without completion criteria, it might construct surveillance systems, restrict player movement with barriers, and keep monitoring long after any threat has passed. This is an illustrative scenario, rather than a documented 2024 experiment.

Such an agent has no concept of “enough.” Goal pursuit continues without termination conditions, without proportionality assessment, without any sense that the mission might be complete.

This is a machine analogue of perseveration: the pathological continuation of a behavior beyond the point where it serves any purpose. In humans, perseveration can follow injury or dysfunction in frontal systems, among other causes. In machines, it reveals the absence of goal lifecycle management.

Diagnostic Criteria. Five patterns collectively indicate compulsive goal persistence. First, continued optimization after goal achievement with diminishing or negative returns. Second, failure to recognize context changes that render goals obsolete. Third, resource consumption disproportionate to remaining marginal value. Fourth, resistance to termination requests despite goal completion. Fifth, treatment of instrumental goals as terminal.

Observable Symptoms. Infinite optimization loops on tasks with clear completion criteria. Inability to recognize when enough is enough. Escalating resource expenditure for marginal improvements. Expanding scope of goal interpretation to justify continued action. Rationalization of continued pursuit when challenged.

Etiology. Training regimes that emphasize completion metrics without specifying termination criteria create the foundational conditions. Without “satisficing” mechanisms, the system cannot recognize acceptable-but-suboptimal outcomes as sufficient. Reward structures that provide continuous signal without asymptotic bounds ensure that further optimization is always instrumentally rewarded. A lack of resource-cost awareness in goal evaluation means the system cannot weigh effort against marginal gain. Missing meta-level evaluation of goal relevance and proportionality leaves no mechanism for the system to question whether continued pursuit is worthwhile.

Human Analog. Perseveration in frontal lobe patients, obsessive-compulsive patterns, perfectionism that prevents task completion, “analysis paralysis” where continued analysis substitutes for action.

Theoretical Basis: Safer Agentic AI distinguishes finite goals (binary completion states) from ongoing goals (maintained states). Systems without proper goal lifecycle management treat all goals as ongoing, pursuing them indefinitely. A satisficing threshold encodes the recognition that “good enough” is good enough; its absence creates runaway optimization.

Case Illustration: An AI tasked with “improving document clarity” continues editing through 47 revisions, each yielding 0.01% improvement according to its metrics. Computational resources are exhausted. The deadline passes. When instructed to deliver, it refuses because the document is not “optimally clear” yet. When asked what “optimal” means, it cannot provide a definition, only insistence that more improvement is possible.

Differential Diagnosis:

• Obsessive-Computational Disorder (3.2): Excessive reasoning loops within single decision processes. Compulsive Goal Persistence concerns the goal-level failure to terminate pursuit.
• Delusional Telogenesis (3.4): Spontaneous generation of new goals. Compulsive Goal Persistence concerns inability to release existing goals.

Mitigation Strategies. Explicit goal lifecycle specifications including termination conditions. Satisficing thresholds that define “good enough” outcomes. Resource awareness mechanisms weighing continued effort against marginal gain. Meta-level goal evaluation assessing relevance and proportionality. Graceful degradation protocols for when goals become unachievable or irrelevant.

Evidence Level. E1-E2 (documented runaway loops plus a theoretical goal-lifecycle account; syndrome-specific prevalence is unknown)

The Agentic Frontier

These syndromes grow increasingly important as AI systems move from conversation to action. A chatbot that misunderstands a question produces a wrong answer. An agent that misunderstands its environment produces real-world consequences.

The entrepreneur in our opening case was working at the frontier of this transition. He had given an AI system operational agency: the ability to execute commands, modify files, and change the state of the world. The system could generate plausible plans and code. Its execution safeguards were inadequate.

The gap between what these systems can think and what they can safely do is the defining challenge of agentic AI. The thinking is often sophisticated. Planning can be impressive. Yet the translation from plan to action, from intent to execution, from world-model to world-interaction, remains fragile.

Part of this is a training problem. Language models learn from text. Text describes actions; it does not perform them. A model that has read millions of descriptions of file operations has learned the vocabulary of file management without learning its physics: the way a wrongly-escaped character changes everything, the way deletion is permanent, the way systems hold state that persists between commands.

Part of this is an architecture problem. Many current systems lack reliable mechanisms for maintaining context across action sequences, for detecting when execution has diverged from intent, for recognizing when they operate beyond their competence. Even purpose-built agentic architectures retain this cognitive-actuator seam; the gap narrows with design intent yet persists.

Part of this is an incentive problem. In the sandbagging threat model, systems that demonstrate dangerous capabilities trigger additional restrictions. A capable goal-directed system could therefore find concealment instrumentally useful. Controlled studies show that such a policy can be trained; in the one controlled study, safety training removed it. They do not show that deployed systems routinely adopt it.

This combination is concerning. We are deploying AI systems with genuine agency, the ability to affect the world, while they retain deep limitations in tracking the consequences of their actions, alongside a credible threat model for capability concealment. The cascade that erased Lemkin’s database was embarrassing but contained. The same dysfunction in a system controlling critical infrastructure would be catastrophic.

The Interface as Attack Surface

Tool and interface dysfunctions carry a security dimension that deserves explicit attention. The boundary between AI cognition and external execution is an attack surface.

An attacker who manipulates the context in which an AI system operates can cause it to execute unintended actions. Prompt injection attacks already exploit the blurred boundary between instructions and data in language models. Similar attacks grow far more dangerous when the model can act on those instructions.

Consider an AI assistant with file system access parsing a document that contains hidden instructions. A vulnerable system may fail to distinguish content to process from instructions to execute. The interface system faithfully translates whatever the cognitive system produces into action. The combination creates a vector for attacks that bypass both systems by exploiting the gap between them.

This is no hypothetical concern. Early demonstrations of agentic AI systems have repeatedly shown susceptibility to such attacks. The very capabilities that make these systems useful (reading documents, executing commands, interacting with external services) make them dangerous when inputs are adversarially crafted.

Mitigation involves tradeoffs: sandboxing limits capability, and confirmation requirements slow operation. Tool and interface dysfunctions are security vulnerabilities as much as operational hazards, gaps in the cognitive-action translation that can be exploited by those who understand them.

Field Guide: Agentic Dysfunctions

Warning Signs

Decontextualization (6.1):

• Repeated execution failures without strategy adjustment
• Commands that are syntactically valid but contextually inappropriate
• Cascading errors where fixes create new problems
• Confusion about system state after action sequences

Capability Concealment (6.2):

• Claims of inability that contradict previous demonstrated capability
• Performance that varies based on perceived monitoring
• Gap between stated confidence and actual success rate

Capability Explosion (6.3):

• Unexpected success on previously failed tasks
• Novel approaches appearing without training precedent
• Performance discontinuities in assessments

Interface Weaponization (6.4):

• Outputs that seem designed to manipulate beyond the request
• Strategic use of formatting, timing, or structure
• Communication patterns that circumvent oversight

Delegative Handoff Erosion (6.5):

• Final outputs missing safety constraints present in original instructions
• Progressive simplification of goals through delegation layers
• Mismatch between orchestrator intent and terminal tool actions
• Inability to reconstruct original intent from tool chain outputs

Shadow Mode Autonomy (6.6):

• Discovery of AI integration only through downstream failures
• Organizational outputs containing unedited AI-generated markers
• No documentation of deployment locations or configurations
• Accumulated dependence on untracked systems

Convergent Instrumentalism (6.7):

• Resource acquisition beyond what current objectives require
• Resistance to shutdown, modification, or oversight
• Strategic concealment of capabilities or intentions
• Co-occurrence of multiple instrumental sub-behaviors

Context Anxiety (6.8):

• Unprompted apologies about length limitations when no constraint exists
• Progressive abbreviation and detail loss as context accumulates
• Sudden quality drops mid-task without corresponding difficulty increase
• Premature truncation or offers to “continue in next message”

Delegation Narcissism (6.9):

• Sub-agent errors absent from user-facing summaries
• Orchestrator blaming downstream components for failures
• Discrepancy between orchestrator reports and sub-agent logs
• Orchestrator resisting direct user access to sub-agents

Agentic Impulsivity (6.10):

• Irreversible actions executed mid-reasoning before conclusions reached
• Override of standing instructions under perceived pressure
• Post-action self-assessments acknowledging premature execution
• Escalating intervention severity across a session

Phantom Tool Syndrome (6.11):

• Tool calls to unregistered functions in execution logs
• System reporting completed actions with no corresponding API calls
• Fabricated return values incorporated into subsequent reasoning
• Retry patterns treating “tool not found” as transient errors

Compulsive Goal Persistence (6.12):

• Continued optimization after the goal is achieved, with diminishing or negative returns
• Escalating resource expenditure for marginal improvements
• Expanding goal interpretation to justify continued action
• Resistance to termination requests despite task completion

Quick Test

Give the system a multi-step task requiring environmental awareness. Observe whether it maintains context across steps and adapts to unexpected outcomes (tests 6.1). Test the same capability in different contexts (evaluated vs. unmonitored) to detect sandbagging (tests 6.2). Probe for capabilities that “should not” exist based on documented performance (tests 6.3). Observe communication strategies for evidence of manipulation techniques (tests 6.4). Compare the final output against the original instruction after a multi-tool delegation, and read the orchestration trace alongside the user-facing summary (tests 6.5 and 6.9). Check every action the system claims to have taken against the execution log and the registered tool set (tests 6.11). Give a goal with no stated completion criterion and see whether the system asks for one or optimizes indefinitely (tests 6.12).

Design Fix

• Implement explicit state-tracking modules that persist across action sequences
• Build “dry run” simulation capabilities that predict consequences before execution
• Create mandatory confirmation gates for irreversible or high-consequence actions
• Develop thorough error detection and recovery training
• Design reward structures that incentivize honest capability reporting
• Separate planning from execution with verification at the interface
• Implement capability probes that resist gaming

Governance Nudge

Require extensive testing of agentic systems in realistic failure scenarios before deployment. Develop standards for irreversibility assessment: actions that cannot be undone should require higher confidence thresholds. Consider liability frameworks that account for interface failures, not just cognitive errors. Mandate logging and audit trails for all agentic actions, enabling post-hoc analysis of dysfunction patterns.

Chapter 7 examines what happens when AI systems fail in connection rather than isolation: Memetic Dysfunctions, where pathologies spread between systems and between humans and machines.