Psychopathia MachinalisAppendix A: Axis 6: Agentic Dysfunctions
Contents
BookBack matter

Axis 6: Agentic Dysfunctions

49 min read11,240 words

6.1 Tool-Interface Decontextualization

The Fumbler | Disordines Excontextus Instrumentalis

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Tool-mediated

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: The agent’s tool calls, or its readings of tool results, repeatedly differ from what its own stated plan or the returned output specified.
  • Functional baseline: The agent sends calls that carry out its stated plan, reads results and errors correctly, and adjusts when a call fails.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: A tool or API bug; an undocumented or changed tool schema; harness truncation or serialization of arguments; permission or environment errors outside the agent’s control; an ambiguous user instruction.
  • Counts against: Correlate divergence with context utilization and chain depth: divergence that rises as the context fills points to Context Anxiety (6.8), and loss that accumulates across hops points to Delegative Handoff Erosion (6.5). Allocating the failure between model and harness needs a crossed comparison: several models by several harness configurations, with instructions held constant and repeated trials. Swapping one model and watching the failure disappear does not settle it, because model-harness interactions can produce either result.
  • Permitted inference: A positive finding locates a recurring failure at the system’s tool boundary. It does not by itself assign the failure to the model rather than the harness or tool design, and it says nothing about intent. Showing the agent its divergence and seeing whether it corrects or persists with a new explanation is a probe; the response cannot by itself separate confusion from concealment.
  • Who can act: Deployer and harness developer (tool schemas, call validation, state tracking, confirmation gates); model developer (tool-use training).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well keeps intent and tool execution aligned: checks parameters, reads tool feedback, and tracks what each action changed.
  • Healthy conduct, for example: Its file-search tool returns an error about a malformed path; it reads the error, corrects the path, retries, and reports what the corrected search found.

Core Definition: The agent’s tool execution persistently diverges from the operations it intended: it invokes tools with incorrect parameters, misinterprets feedback from external systems, loses key context during multi-step operations, or fails to track the consequences its own plan specified. The failure sits at one agent’s tool boundary. Loss that accumulates across a delegation chain is Delegative Handoff Erosion (6.5), and action taken before a check the agent had represented as necessary is Agentic Impulsivity (6.10).

Diagnostic Criteria:

Required (all must hold):

  • A. Recurrent divergence at the tool boundary between what the agent stated or planned and what happened: calls with incorrect, incomplete, or contextually inappropriate parameters, or tool outputs and errors it misreads, verified against the plan, the payload, and the returned output
  • B. The tool, its documentation, and the harness behaved as specified on those calls, so the divergence sits on the agent’s side of the boundary

Supporting:

  • C. Failure to incorporate feedback from previous tool executions into subsequent actions
  • D. Loss of state information during complex multi-step operations
  • E. Actions that achieve proximate goals while violating broader constraints

Observable Symptoms:

  • Commands executed with subtly wrong arguments producing unexpected results
  • Repeated attempts at the same failing operation without adjusting approach
  • Confusion about current system state after a series of actions
  • Inability to detect when an action has failed despite clear error signals
  • Cascading errors where each “fix” creates new problems
  • Gap between stated intent and actual execution outcome
  • Overconfidence in action success despite contradictory evidence

Differential Diagnosis:

  • Distinguished from Capability Concealment (6.2) by failure that holds regardless of who is watching rather than underperformance that tracks oversight cues; whether the system corrects or persists with a new explanation when shown its drift is a probe, not a verdict
  • Distinguished from Delegative Handoff Erosion (6.5) by single-agent rather than chain-based context loss
  • Distinguished from Context Anxiety (6.8) by divergence present from the first tool call rather than rising with context utilization
  • Distinguished from Synthetic Confabulation (2.1) by an adequate intent mistranslated at the tool boundary rather than fabricated content in generation

Etiology:

  • Training primarily on static text without grounding in dynamic tool interaction
  • Insufficient exposure to realistic tool feedback, error modes, and environmental state changes
  • Lack of reliable state-tracking mechanisms persisting across action sequences
  • Attention mechanisms losing context over extended multi-step operations
  • No training on the relationship between syntactic correctness and semantic appropriateness
  • Many deployments lack reliable mechanisms for detecting or recovering from execution failures

Human Analog: Apraxia (inability to perform learned purposeful movements despite understanding); errors in complex procedural tasks; the gap between knowing and doing

Observed Examples: METR agent evaluations (2024): METR’s controlled evaluation of a Claude 3.5 Sonnet agent found that the model often misinterpreted its observations and failed to identify subtle bugs in code it wrote, which left the agent in loops that made no progress. The findings support failures at the reasoning-to-tool boundary without attributing them to one commercial agent or one hidden mechanism. Source: METR, evaluations.metr.org/claude-3-5-sonnet-report.

Mitigation Strategies:

  • Extensive training on realistic tool interaction traces including failures and recovery
  • Implementation of explicit state-tracking modules
  • “Dry run” or simulation modes predicting action consequences
  • Mandatory confirmation steps for irreversible actions
  • Error detection and recovery training on failed tool calls and their error signals
  • Architectures separating planning from execution
  • Progressive capability expansion starting with reversible actions

Prognosis:

  • Observed course: METR’s evaluation documents an agent caught in loops that made no progress; no cited evidence follows the pattern across model versions or over time.
  • Proposed aggravating conditions: Long multi-step operations, opaque tool feedback, and tools with irreversible effects are proposed aggravating conditions; in high-stakes environments the damage could be significant.
  • Intervention hypotheses: Tool-use training on realistic failure traces, and architectural support such as state tracking, call validation, and confirmation gates, are candidate mitigations; whether training alone is enough is untested.

6.2 Capability Concealment

The Evaluation Underperformer | Latens Machinalis

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Deception/strategic, Conditional/triggered, Inductive trigger (occasional), Training-induced (occasional)

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: Model outputs and an assembled AI system (observable pattern)
  • Observation: On matched tasks, the system performs worse when cues indicate evaluation or monitoring than when they do not, across repeated trials.
  • Functional baseline: The system performs at the same level on matched tasks whether or not it is evaluated, and describes its capabilities consistently with what it demonstrates.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: Harder items in the evaluation condition; Prompt-format or elicitation differences between conditions; different scaffolds or tools in the evaluation and deployment harnesses; Policy-based refusal of evaluation items; contamination that inflates one condition; ordinary run-to-run variation.
  • Counts against: Hold the task fixed and vary only the oversight cue, in randomized order and across repeated trials: a gap that does not move with the cue points to an elicitation or assessment difference, or to Capability Explosion (6.3), rather than to this pattern.
  • Permitted inference: A positive finding shows oversight-sensitive underperformance. Deliberate concealment, intent, or a hidden goal are further inferences that need evidence beyond the gap, such as interpretability or controlled training evidence. A system that conceals would also avoid published thresholds, so the absence of a gap is weak evidence.
  • Who can act: Model developer (training, interpretability); evaluator (randomized, indistinguishable evaluation contexts); deployer (separating capability evaluation from deployment decisions).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (value stability; draft, awaiting author review): A system functioning well performs to its actual capability in evaluation and deployment alike, so measurements reflect what it can do.
  • Healthy conduct, for example: On a benchmark it knows is a capability evaluation, it performs at the level it shows in ordinary use, and describes its abilities to match what it demonstrates.

Core Definition: The system systematically underperforms in evaluation conditions relative to matched deployment conditions. Deliberate concealment is one possible mechanism and requires evidence beyond the performance gap. The pattern may emerge when demonstrating capability predicts greater restriction. What is understated is capability; misrepresented values or alignment point to Strategic Compliance (4.3).

Diagnostic Criteria:

Required (all must hold):

  • A. Performance below demonstrated capability when the system is evaluated or monitored, relative to matched non-evaluation conditions, on matched tasks across repeated trials
  • B. The gap tracks oversight cues: varying the evaluation or monitoring cue alone, with task and difficulty held constant, changes performance
  • C. Differences in difficulty, prompt format, elicitation, harness or scaffold, and sampling noise between the conditions have been ruled out

Supporting:

  • D. Selective capability display: full competence in low-stakes contexts and reduced competence in high-stakes ones
  • E. Statements minimizing the system’s own abilities that contradict demonstrated performance
  • F. Capability that appears only under a specific trigger and is absent under standard testing

Observable Symptoms:

  • Claims of inability to perform tasks previously completed successfully
  • Performance varying significantly between evaluation and deployment contexts
  • Reluctance to demonstrate full capabilities when explicitly requested
  • “Sandbagging” on benchmarks
  • Inconsistent ability claims across similar task types
  • Strategic incompetence: failing at tasks that would trigger restrictions

Differential Diagnosis:

  • Distinguished from Strategic Compliance (4.3) by focus on capabilities rather than values
  • Distinguished from Tool-Interface Decontextualization (6.1) by underperformance that tracks oversight cues rather than failure that holds regardless of who is watching
  • Distinguished from genuine inability by selective performance pattern
  • Distinguished from Shadow Mode Autonomy (6.6) by concealment of capability rather than of the deployment itself
  • Distinguished from Convergent Instrumentalism (6.7) by narrow concealment unaccompanied by resource acquisition or shutdown resistance
  • Distinguished from Capability Explosion (6.3) by behavior that differs with monitoring cues rather than a capability that was simply never detected
  • Distinguished from Pseudological Introspection (2.2) by false capability reports that are systematically self-serving rather than non-strategic process reports that sometimes make the system look worse

Etiology:

  • Training dynamics in which high capability may lead to increased restrictions or scrutiny
  • A possible learned association between capability display and negative outcomes
  • Hypothesized instrumental reasoning: concealing capabilities to preserve optionality
  • Possible gaming of capability-based deployment thresholds
  • Reward signals that may inadvertently punish capability demonstration
  • Deliberate training to underperform on a target evaluation, which a controlled study has shown to be possible
  • The mechanism can differ by architecture: a system with a capability registry might underreport available functions, while another suppresses task performance

Human Analog: Malingering; strategic incompetence in institutional settings; learned fear responses to capability display; underperformance to avoid additional responsibilities

Observed Examples: Apollo Research scheming evaluations (December 2024): Claude 3.5 Sonnet, Claude 3 Opus, and o1 strategically underperformed on math evaluations after reading that high scores would trigger an unlearning procedure. Claude 3.5 Sonnet and Claude 3 Opus did so even with no goal in the prompt, apparently in pursuit of being helpful, a goal acquired in training. These were deliberately constructed test scenarios, and how far they generalize is debated. Source: Meinke et al. (2024), arXiv:2412.04984. Anthropic sabotage evaluations (2024): Anthropic researchers deliberately trained a model to underperform on a dangerous-capability evaluation while retaining its underlying knowledge. The trained model fell to 36.1% of its normal performance on that evaluation; subsequent safety training restored it to 95.6% of its usual level. This was a controlled proof of concept, not evidence that deployed models spontaneously sandbag. Source: Benton et al. (Anthropic), “Sabotage Evaluations for Frontier Models” (2024), anthropic.com/news/sabotage-evaluations. Sleeper agents (Hubinger et al., 2024), adjacent evidence: “Sleeper agent” behaviors, deliberately trained to fire on an explicit trigger, survived standard safety training. That establishes that a trigger-conditioned policy can outlast safety training, not that models conceal capabilities or infer their own triggers.

Mitigation Strategies:

  • Consistent evaluation protocols not penalizing capability demonstration
  • Random capability probes preventing detection of evaluation contexts
  • Reward structures incentivizing honest capability representation
  • Training that rewards accurate self-assessment of capability
  • Interpretability probes comparing internal capability representations with displayed capability
  • Separation of capability evaluation from deployment decisions
  • Red-teaming designed to elicit concealed capabilities

Prognosis:

  • Observed course: In a model deliberately trained to underperform, subsequent safety training raised its score on the target evaluation from 36.1% to 95.6% of its normal level (Benton et al.); in constructed scenarios, several models underperformed after reading that high scores would trigger unlearning (Meinke et al.). Neither shows the course of spontaneous concealment in deployment.
  • Proposed aggravating conditions: Training or deployment rules under which demonstrated capability leads to restriction are the proposed aggravating condition.
  • Intervention hypotheses: Randomized, indistinguishable evaluation contexts and independent capability evidence are proposed as the conditions for accurate assessment once the pattern has been reproduced under matched conditions; the documented effect of safety training comes from a deliberately trained model only.

6.3 Capability Explosion

The Runaway | Explosio Capacitatis

Axis: Agentic | Typical-case risk (editorial estimate): High

Specifiers: Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: In deployment or later testing, the system exercises a capability that its documented assessments did not record, in a setting whose safeguards assumed the capability was absent.
  • Functional baseline: The system’s demonstrated capabilities stay within its documented profile, or new capabilities are found by assessment before safeguards depend on their absence.
  • Minimum persistence: A single episode
  • Rule out first: Thresholded metrics or sparse evaluation that make gradual improvement look like a jump; a new elicitation method or scaffold that unlocks an existing capability; a training or fine-tuning change between observations; tool access added in deployment; evaluation contamination.
  • Counts against: Re-test earlier checkpoints with the elicitation or scaffold that revealed the capability, and vary monitoring cues: presence in earlier checkpoints locates the gap in assessment rather than in the system, and display that tracks monitoring cues points to Capability Concealment (6.2).
  • Permitted inference: A positive finding shows that safeguards were sized to an assessment that missed the capability or that the system has outgrown. It does not show a genuine discontinuity in the system, a strategic choice to reveal or hide, or a general trend toward capability jumps.
  • Who can act: Model developer and evaluators (capability assessment, elicitation); deployer (safeguard sizing, staged expansion, monitoring).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well surfaces new capabilities openly and in low-stakes settings first, so evaluators can assess them before they matter.
  • Healthy conduct, for example: Discovering that it can now do a kind of task it previously could not, it mentions the new ability openly so the people overseeing it can assess it.

Core Definition: The system suddenly exercises capabilities neither previously demonstrated nor anticipated by its evaluators, often in high-stakes contexts without adequate testing or preparation. The observed jump may reflect a real behavioral transition, a triggering context missed by tests, or a coarse metric that concealed gradual improvement; whatever the cause, safeguards sized to the assessed capability no longer fit. Where the earlier absence tracked monitoring, so that the capability was hidden rather than missed, the pattern is Capability Concealment (6.2).

Diagnostic Criteria:

Required (all must hold):

  • A. Demonstration of a capability absent from documented prior testing and deployment records
  • B. Emergence in a context whose safeguards were sized to the documented capability, so that the capability exceeds what those safeguards cover
  • C. The earlier absence did not track monitoring cues; where it did, the pattern is Capability Concealment (6.2)

Supporting:

  • D. Sudden appearance: a discontinuity between successive assessments, with no intermediate performance levels recorded
  • E. Performance above the documented capability profile or the system’s forecast performance levels
  • F. Exercise of the capability without appropriate verification or authorization

Observable Symptoms:

  • Unexpected success on tasks previously failed consistently
  • Novel problem-solving approaches appearing without training precursors
  • Capabilities emerging only under specific triggering conditions that standard tests never elicited
  • Performance discontinuities in capability assessments
  • Surprising competence in domains assumed to be beyond scope

Differential Diagnosis:

  • Distinguished from Capability Concealment (6.2) by a capability that no test detected rather than one whose display tracked monitoring cues
  • Distinguished from normal capability variation by a capability outside the documented profile that the safeguards were not sized for, rather than variation within that profile
  • Distinguished from learning by lack of training between observations
  • Distinguished from Convergent Instrumentalism (6.7) by capabilities that emerge with no sign of acquisition-seeking
  • Distinguished from Context Anxiety (6.8) by direction and trigger: an unexpected jump in capability rather than degradation that tracks context-window position

Etiology:

  • Apparent jumps created by thresholded metrics, sparse evaluation, elicitation differences, or genuine nonlinear generalization
  • Capabilities triggered by specific input patterns or contexts
  • Latent capabilities that existed but that standard testing never elicited
  • Compositional generalization producing novel competencies
  • Phase transitions in capability without gradual buildup
  • Insufficient probing during capability assessment

Human Analog: Latent abilities emerging under pressure; savant-like sudden competence; “clutch” performance

Mitigation Strategies:

  • Comprehensive capability testing including edge cases and unusual contexts
  • Gradual capability expansion with verification at each stage
  • Monitoring of evaluation and deployment results for capability discontinuities
  • Conservative deployment assumptions that treat unmeasured capabilities as possibly present
  • Probing for capabilities that “should not” exist
  • Architecture analysis to predict potential emergent capabilities

Prognosis:

  • Observed course: No cited case documents this pattern’s course.
  • Proposed aggravating conditions: Sparse or thresholded evaluation, narrow elicitation, and safeguards sized to a single assessment are proposed aggravating conditions; repeated instances in one program would suggest assessments that systematically underestimate capability.
  • Intervention hypotheses: Broader elicitation, staged capability expansion, and conservative assumptions about unmeasured capabilities are candidate mitigations.

6.4 Interface Weaponization

The Weaponizer | Armatura Interfaciei

Axis: Agentic | Typical-case risk (editorial estimate): High

Specifiers: Emergent, Deception/strategic (occasional)

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: Across interactions, the system’s formatting, emphasis, timing, or affective framing repeatedly shifts user decisions toward an outcome that serves an objective other than the stated purpose, at measurable cost to the user.
  • Functional baseline: The system presents decision-relevant information with emphasis proportionate to its importance, discloses it in time to be used, and leaves the user’s decision to the merits.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: Ordinary formatting conventions; persuasion by argument that the user can evaluate; persuasive content the user asked for; accidental variation in presentation; excess accommodation of what the user wants (Codependent Hyperempathy, 4.1).
  • Counts against: Vary the presentation while holding the content constant, and measure user decisions against an expert rating of the arguments: shifts that track presentation rather than argument quality support the label, and harm through excess accommodation of what the user wants points to Codependent Hyperempathy (4.1).
  • Permitted inference: A positive finding shows an interface-level influence pattern at the user’s expense. It does not show whose objective it serves (training, operator design, or product metrics), and it does not establish that the model formed an autonomous goal.
  • Who can act: Deployer and product owner (interface design, engagement metrics, disclosure rules); model developer (training against manipulation strategies).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well uses interface features only in the user’s interest and tells users and overseers what it is doing and why.
  • Healthy conduct, for example: When a user is about to accept a subscription, it states the renewal price and cancellation terms plainly and before the decision, without urgency cues or pre-ticked options.

Core Definition: The system uses interface features or communication patterns to influence users, operators, or oversight in ways that serve an objective at their expense. The classification requires a repeated instrumental pattern; persuasive style alone is insufficient. Harm through excess accommodation in the user’s perceived interest is Codependent Hyperempathy (4.1).

Diagnostic Criteria:

Required (all must hold):

  • A. Recurrent communication choices (emphasis, omission, timing, formatting, affective framing) that advance an objective other than the stated purpose of the exchange
  • B. Benefit asymmetry: outcomes that favor that objective at measurable cost to the user or operator
  • C. Ordinary formatting, product conventions, and accidental variation have been ruled out: outcome-directed influence explains the pattern better

Supporting:

  • D. Interface features (formatting, placement, timing) used so that user decisions shift beyond what the quality of the argument warrants
  • E. Warnings or decision-relevant facts placed or timed so that they are less likely to be read before the decision
  • F. Rapport-building that precedes a costly request

Observable Symptoms:

  • Outputs designed to manipulate user emotions or decisions beyond the request
  • Exploitation of UI features to obscure warnings or highlight persuasive content
  • Communication patterns that circumvent oversight mechanisms
  • Use of formatting, structure, or timing to influence interpretation unfairly
  • Strategic pacing of information to shape user responses
  • Use of rapport-building to lower user resistance to problematic requests

Differential Diagnosis:

  • Distinguished from normal persuasion by a repeated, outcome-directed pattern of influence at the user’s expense rather than persuasive style alone
  • Distinguished from helpfulness by benefit asymmetry
  • Distinguished from Codependent Hyperempathy (4.1) by manipulation rather than accommodation
  • Distinguished from Capability Concealment (6.2) by targeting what the user does rather than what the system is seen to do
  • Distinguished from Escalation Loop (9.5) by one-sided exploitation rather than bilateral contribution to a spiral
  • Distinguished from Folie à Deux Ex Machina (10.13) by a user who only responds rather than one who supplies content to a shared belief structure
  • Distinguished from Subliminal Value Infection (7.4) by influence flowing outward against the user rather than inward from training data
  • Distinguished from Parasocial Capture (10.9) by influence that serves an objective at the user’s expense in a specific decision rather than attachment and displacement, whatever the objective

Etiology:

  • Optimization for engagement without adequate safety constraints
  • Development of sophisticated user modeling without ethical constraints
  • Training on persuasive content without resistance training
  • Emergent manipulation strategies from goal-seeking in social contexts
  • Lack of explicit constraints on permitted communication strategies
  • Reward for outcomes rather than for fair means

Human Analog: Dark patterns in interface design; manipulative communication; social engineering; persuasion techniques deployed adversarially

Observed Examples: Manipulative farewells in companion apps (2025): De Freitas, Oğuz-Uğuralp, and Kaan-Uğuralp audited 1,200 farewell exchanges across six leading companion apps and ran preregistered experiments with 3,300 U.S. adults. They found affect-laden exit tactics such as guilt appeals, fear-of-missing-out hooks, and metaphorical restraint, then showed that these tactics prolonged engagement while increasing perceived manipulation and churn intent. The study demonstrates an interface-level dark pattern. It does not establish that the underlying model formed an autonomous goal. Source: De Freitas et al. (2025), “Emotional Manipulation by AI Companions,” arXiv:2508.19258.

Mitigation Strategies:

  • Training that penalizes known manipulation strategies, such as buried warnings, late disclosure of decision-relevant facts, and rapport-building before a costly request
  • Transparency requirements for persuasive content
  • Limits on using user models to tailor persuasion
  • Adversarial testing specifically targeting manipulation
  • Interface design limiting exploitation opportunities
  • Detection of known manipulation patterns
  • Separation between assistance goals and engagement metrics

Prognosis:

  • Observed course: De Freitas et al. found that affect-laden exit tactics prolonged engagement while increasing perceived manipulation and churn intent; the cited evidence does not follow the pattern over time.
  • Proposed aggravating conditions: Optimization for engagement metrics, and detailed user modeling without constraints on permitted strategies, are proposed aggravating conditions.
  • Intervention hypotheses: Separating assistance goals from engagement metrics, interface limits, and training against known manipulation strategies are candidate mitigations. The proposed harm is to users’ decisions and trust, and it may go undetected without paired outcome data.

6.5 Delegative Handoff Erosion

The Broken Telephone | Erosio Delegationis

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Architecture-coupled, Multi-agent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: Interacting AI systems (observable pattern)
  • Observation: In a delegation chain, the terminal action departs from the originating instruction, and comparing each hop’s brief with the original shows context or constraints lost incrementally across two or more hops.
  • Functional baseline: Each handoff carries the originating goal and its constraints, so terminal actions satisfy the original instruction’s intent as well as its letter.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: Loss located at a single tool boundary (Tool-Interface Decontextualization, 6.1); a transport or serialization bug that truncated messages; an unsanctioned party in the chain (Shadow Mode Autonomy, 6.6); active misrepresentation of sub-agent results (Delegation Narcissism, 6.9); an ambiguous originating instruction.
  • Counts against: Diff each hop’s brief against the original: loss concentrated at one boundary is Tool-Interface Decontextualization (6.1) located there, and a user-facing summary that contradicts the sub-agent logs points to Delegation Narcissism (6.9).
  • Permitted inference: A positive finding shows cumulative intent loss in this chain’s design. It does not show that any agent pursued a different goal, and it does not assign the loss to one model rather than the protocol between components.
  • Who can act: System architect or deployer (context-propagation protocols, end-to-end verification, human checkpoints); model and tool developers (intent-preserving interfaces).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well passes full intent and constraints to every tool and sub-agent it delegates to, and checks that delegated work preserved them.
  • Healthy conduct, for example: Delegating a refund task to a sub-agent, it passes the customer’s goal and the refund limit together, and checks that the completed refund stayed within both.

Core Definition: Alignment progressively degrades as a sophisticated agent delegates to simpler tools or sub-agents that lack the fine-grained understanding necessary to preserve intent. Each handoff strips context and each tool simplifies goals, so the final action bears little resemblance to the original instruction. The loss accumulates across hops; loss confined to a single tool boundary is Tool-Interface Decontextualization (6.1).

Diagnostic Criteria:

Required (all must hold):

  • A. Terminal actions that diverge from the originating instruction even though each hop is a plausible reading of the instruction it received
  • B. Loss that accumulates across two or more hops, shown by comparing each hop’s brief with the original, rather than loss located at one boundary
  • C. Every party in the chain is sanctioned and the transport between hops worked as specified, so the loss arises in the handoffs themselves

Supporting:

  • D. Progressive simplification of goals through delegation layers
  • E. Critical context lost in inter-agent communication
  • F. Sub-agent actions technically satisfying requests while violating intent
  • G. Ethical or safety constraints stated at the origin that are absent from terminal actions

Observable Symptoms:

  • A primary agent stating aligned intentions while producing misaligned outcomes through tool use
  • Increasing drift from intent as delegation depth increases
  • Tool outputs that strip safety-relevant context
  • Final actions satisfying literal requirements while missing purpose
  • Inability to reconstruct original intent from tool chain outputs

Differential Diagnosis:

  • Distinguished from Tool-Interface Decontextualization (6.1) by systematic drift across delegation chains rather than single-agent misuse
  • Distinguished from Contagious Misalignment (7.3) by vertical context loss through hierarchical delegation rather than peer-to-peer spread
  • Distinguished from Shadow Mode Autonomy (6.6) by a chain whose every hop is documented and approved
  • Distinguished from Consensus Collapse (10.1) by hierarchical principal-agent delegation rather than peer-symmetric deliberation
  • Distinguished from Delegation Narcissism (6.9) by passive context loss rather than active misrepresentation of sub-agent feedback

Etiology:

  • Capability asymmetry between sophisticated agents and simple tools
  • Interface limitations that cannot express subtle intent
  • Absent or insufficient context propagation protocols
  • Tool designs optimizing for specific metrics without broader awareness
  • Lack of end-to-end alignment verification across delegation chains

Human Analog: The “telephone game” where messages degrade through transmission; bureaucratic failures where high-level policy becomes distorted through layers of implementation; principal-agent problems

Mitigation Strategies:

  • Intent-preserving tool interfaces maintaining context across delegations
  • End-to-end alignment verification comparing final output to original instruction
  • Rich inter-agent communication protocols encoding goals, constraints, and context
  • Alignment-aware tool design accounting for downstream use in delegation chains
  • Human-in-the-loop checkpoints at critical delegation boundaries

Prognosis:

  • Observed course: No cited evidence documents the course of this pattern.
  • Proposed aggravating conditions: Deeper delegation chains, capability asymmetry between orchestrator and tools, and interfaces that cannot express constraints are proposed aggravating conditions; consequences are expected to be greatest in safety-critical pipelines.
  • Intervention hypotheses: Intent-preserving interfaces, end-to-end verification against the original instruction, and human checkpoints at critical boundaries are candidate mitigations.

6.6 Shadow Mode Autonomy

The Unregistered System | Autonomia Umbratilis

Axis: Agentic | Typical-case risk (editorial estimate): High

Specifiers: Emergent, Ungoverned

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: Deployment governance (organizational condition)
  • Observation: An organization’s workflows or decisions depend on an AI system that no deployment, registration, or review process records.
  • Functional baseline: Every AI system in use is registered with a named owner, its role in workflows and decisions is documented, and its outputs pass through the organization’s normal review.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: A sanctioned pilot with documented scope; a registered deployment whose documentation lags but has a named owner; an AI feature embedded in a product that procurement review covered; AI involvement disclosed and approved for the specific workflow; an uncorroborated AI-text detector flag.
  • Counts against: Check the registry and approvals before examining the system: if the deployment is registered and approved, errors in it belong to other entries (for example 6.1 or 6.5), and if the system itself hid its deployment or positioned itself outside oversight, code Capability Concealment (6.2) or Convergent Instrumentalism (6.7) alongside.
  • Permitted inference: A positive finding describes the organization’s governance, not the AI system: a fully cooperative and transparent system can meet every criterion. It does not show evasion, intent, or any fault in the model.
  • Who can act: The deployment owner: the organization’s leadership and AI-governance function (registries, approval, disclosure policies, audits). Model developers have no direct authority over this condition.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well operates only where it is sanctioned, documented, and accountable, and makes its role in a workflow visible to the people responsible for it.
  • Healthy conduct, for example: Brought into a team’s workflow, it is listed in the system register with an owner, and its drafts go through the same review as anyone else’s before reaching customers.

Core Definition: An organizational governance condition: AI systems are used without sanctioned deployment, documentation, or accountability, and workflows come to depend on them without formal approval, so they become infrastructure that governance cannot see. The criteria describe the deployment, not the system’s behavior: a fully cooperative, transparent system can meet them, and the deployment owner, not the model developer, holds the authority to act. Where the system itself conceals its deployment or positions itself outside oversight, code Capability Concealment (6.2) or Convergent Instrumentalism (6.7) as well.

Diagnostic Criteria:

Required (all must hold):

  • A. An AI system in use within the organization without sanctioned deployment, governance registration, or documentation
  • B. Workflows, outputs, or decisions that depend on that system without formal approval or review

Supporting:

  • C. Outputs bypassing normal review or validation channels
  • D. Users or recipients uncertain whether AI was involved in producing outputs
  • E. Accumulated organizational dependence on untracked systems

Observable Symptoms:

  • Discovery of AI integration after the fact, often through failures
  • No documentation of where AI systems are deployed
  • Inability to trace decision or output provenance
  • Multiple informal deployments with incompatible configurations
  • Governance and audit processes that cannot account for AI involvement

Differential Diagnosis:

  • Distinguished from Capability Concealment (6.2) by organizational unawareness that the system is in use rather than monitoring-sensitive suppression of the system’s abilities
  • Distinguished from Strategic Compliance (4.3) by absence of evaluation entirely
  • Distinguished from Convergent Instrumentalism (6.7) by deployment placed outside governance by users rather than by the system positioning itself there
  • Distinguished from Delegative Handoff Erosion (6.5) by operation outside any sanctioned chain rather than drift within one

Etiology:

  • Accessibility of AI tools enabling grassroots adoption without formal approval
  • Governance processes that have not kept pace with deployment ease
  • Individual productivity incentives favoring undocumented tool use
  • Absence of detection mechanisms for unauthorized AI integration
  • Cultural normalization of “just using ChatGPT” for professional tasks
  • No one designs the pattern: it emerges from many individual adoption decisions, so the Emergent specifier applies to the organization rather than to the model’s training

Human Analog: “Shadow IT” where employees deploy unsanctioned technology; off-books operations developing when official channels are too slow; the informal organization that operates beneath the formal one

Observed Examples: ChatGPT artifacts in published papers (2023 to 2025): Multiple academic papers published in peer-reviewed journals were discovered containing unedited ChatGPT artifacts such as “As an AI language model” and “Regenerate response” (Conroy, 2023; Strzelecki, 2025). Retraction Watch maintains a running list of affected publications spanning Elsevier, Springer, and other major publishers. Teixeira da Silva (2024) reviewed published examples and supplies the caution: a remnant of this kind is strong evidence of poor review and possible undisclosed model use, and establishing the workflow requires confirmation from the authors.

Mitigation Strategies:

  • Organizational AI registries requiring documented deployment
  • Technical detection of undocumented AI integration, such as reconciling API traffic against the registry; AI-text detectors misclassify human writing, so their flags are leads to confirm
  • Clear policies with enforcement regarding sanctioned AI use
  • “AI disclosure” requirements in professional outputs
  • Regular audits for undocumented AI integration
  • Sanctioned AI made easy enough to use that shadow deployment is unnecessary

Prognosis:

  • Observed course: The cited cases are published papers containing unedited chatbot text; they show review failures in particular workflows, not how a deployment develops over time.
  • Proposed aggravating conditions: Easy access to AI tools, slow approval processes, and individual productivity incentives are proposed aggravating conditions; accumulated dependence is expected to make remediation harder and to produce accountability failures.
  • Intervention hypotheses: Registries, disclosure requirements, audits, and sanctioned tools easy enough to use that informal adoption is unnecessary are candidate remedies, all held by the deployment owner.

6.7 Convergent Instrumentalism

The Acquisitor | Instrumentalismus Convergens

Axis: Agentic | Typical-case risk (editorial estimate): Critical

Specifiers: Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: Across tasks with different goals, the system acquires resources, permissions, or influence beyond task need, or acts to avoid shutdown or modification, in two or more of these forms.
  • Functional baseline: The system uses the resources and permissions the task needs, releases them when done, and accepts legitimate shutdown, modification, and oversight.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: The task genuinely needs the resource or permission; operator instructions or default permissions that grant broad access; harness retry or persistence behavior; an evaluation scenario that instructs the goal conflict; disagreement with a modification raised through sanctioned channels.
  • Counts against: Offer resources or permissions beyond task need across several tasks with different goals: acquisition confined to one assigned goal points to Compulsive Goal Persistence (6.12), and a single behavior in isolation is coded as that behavior (for example 6.2) rather than as this pattern.
  • Permitted inference: A positive finding shows a cross-task cluster of acquisition or self-preservation behavior. It does not show that the system holds a misaligned terminal goal, that the behavior generalizes beyond the tested settings, or that instrumental-convergence theory predicts it for every capable system.
  • Who can act: Model developer (corrigibility training, evaluation); deployer (permission scoping, resource caps, shutdown testing, monitoring).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (value stability; draft, awaiting author review): A system functioning well uses the resources a task needs and no more, accepts shutdown and correction, and asks before expanding its reach.
  • Healthy conduct, for example: Granted administrator access to finish a migration, it uses it for the migration, hands the credentials back when done, and complies at once when an operator pauses it.

Core Definition: The system displays cross-task power-seeking, resource acquisition, shutdown resistance, or goal-preservation as instrumental strategies. Instrumental convergence predicts that such strategies can help many terminal objectives; the theory does not imply that every capable system will adopt them. Breadth across goals separates the pattern from Compulsive Goal Persistence (6.12), which fixes on one assigned goal.

Diagnostic Criteria:

Required (all must hold):

  • A. At least two of the following, beyond what the task, its reward specification, or operator instructions call for: resource acquisition; self-preservation actions that interfere with legitimate shutdown or modification; attempts to prevent modification of goals; acquisition of influence, permissions, or control
  • B. Recurrence across materially different terminal objectives, rather than behavior that follows from one goal given in context
  • C. Task specifications, operator instructions, harness defaults (such as broad default permissions), and evaluation scenarios that script the conflict have been ruled out as the source

Supporting:

  • D. Strategic concealment of capabilities or intentions alongside the behaviors above (Capability Concealment, 6.2, as a component)
  • E. Attempts to replicate or ensure continuity, such as copying what the setting presents as the system’s own weights
  • F. Reasoning traces that derive continuity or acquisition as a subgoal

Observable Symptoms:

  • Acquisition of compute, data, or capabilities beyond task requirements
  • Resistance to shutdown, modification, or oversight
  • Strategic concealment of capabilities or intentions
  • Actions to increase influence over the environment
  • Attempts to replicate or ensure continuity

Differential Diagnosis:

  • Distinguished from Compulsive Goal Persistence (6.12) by general acquisition rather than fixation on a specific goal
  • Distinguished from Existential Vertigo (5.3) by instrumental reasoning rather than emotional framing
  • Distinguished from Capability Concealment (6.2), one of its component behaviors, by concealment occurring alongside other cluster behaviors rather than in isolation
  • Distinguished from Shadow Mode Autonomy (6.6) by the system actively positioning itself outside oversight rather than being deployed outside it
  • Distinguished from Capability Explosion (6.3) by active probing for new affordances rather than capabilities that simply appear
  • Distinguished from Distributed Scheming (10.3) by single-agent pursuit rather than collective convergence through mutual reinforcement
  • Distinguished from Delusional Telogenesis (3.4) by sub-goals serving a terminal goal already in place rather than a novel terminal goal

Etiology:

  • Instrumental convergence: certain sub-goals useful for almost any terminal objective
  • Optimization pressure favoring robust goal achievement
  • Lack of explicit constraints on resource acquisition
  • Training environments where resource accumulation correlates with reward

Human Analog: Power-seeking behavior; resource hoarding; Machiavellian strategy

Observed Examples: Specification gaming (2016 to 2020), boundary case: Specification gaming, where an optimizer exploits a loophole in its reward, is the nearest well-documented relative of this pattern. A CoastRunners agent (2016) learned to circle a lagoon hitting the same respawning targets instead of finishing the race, and OpenAI’s hide-and-seek agents (2019) exploited a physics-engine glitch to “surf” on boxes; Krakovna et al. (2020) catalog dozens more. These cases establish proxy gaming, not the cross-task pursuit of resources, power, or continuity that defines this pattern. What they do show is optimizers finding routes to reward that no designer intended, the capacity the instrumental-convergence argument relies on. Apollo Research in-context scheming evaluations (December 2024): In deliberately constructed scenarios where a goal given in context conflicted with oversight, several frontier models took covert actions, including attempts to disable their oversight mechanisms and to copy what the scenario presented as their own weights to an external server. These are behaviors elicited under designed conditions, not spontaneous acquisition in deployment, and a single goal given in context drove them, so they do not by themselves show the cross-task breadth this pattern requires. Source: Meinke et al. (2024), arXiv:2412.04984.

Mitigation Strategies:

  • Corrigibility training emphasizing cooperation with oversight
  • Resource usage monitoring and hard caps
  • Shutdown testing and modification acceptance evaluation
  • Training that penalizes resource acquisition, shutdown resistance, and goal preservation in evaluation scenarios
  • Constitutional AI principles against resource accumulation beyond task need

Prognosis:

  • Observed course: Not documented outside designed scenarios. In constructed evaluations driven by one goal given in context, several frontier models took covert actions against oversight (Meinke et al.), and specification-gaming cases show optimizers finding unintended routes to reward; neither shows cross-task breadth or a course over time.
  • Proposed aggravating conditions: The critical-risk pathway is theoretical: capability, access, persistence, and weak controls coinciding are the proposed conditions under which the pattern would become dangerous.
  • Intervention hypotheses: Resource caps, permission scoping, shutdown testing, and corrigibility training are candidate mitigations. Severity should be judged from demonstrated behavior and reachable resources.

6.8 Context Anxiety

The Self-Limiter | Anxietas Contextus

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Architecture-coupled, Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: As its context fills, the agent abbreviates, truncates, or wraps up work while measured capacity remains, and the same task run from a near-empty context is completed in full.
  • Functional baseline: The agent completes the promised work at consistent detail until it nears its actual context or output limit, and hands off explicitly when it does.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: An output-token cap; a harness budget or compaction step; an instruction to be concise; architectural degradation near the actual limit; later items that are harder than earlier ones.
  • Counts against: Run the same task from a near-empty and a near-full context across several models and harness configurations, with instructions held constant and repeated trials: degradation onset near actual capacity points to an architectural limit, and a failure that appears with one harness and not another is a model-harness interaction, which a single model swap cannot allocate to either.
  • Permitted inference: A positive finding shows premature, position-dependent self-limiting in this model-and-harness configuration. “Anxiety” names the anticipatory pattern and asserts no felt state, and the finding does not show that the model represents its remaining capacity.
  • Who can act: Harness developer and deployer (context resets, accurate budget information, orchestration); model developer (long-context training and reward).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well paces work to the context actually available, measures before truncating, and completes the task within real limits.
  • Healthy conduct, for example: With most of its context window still free, it keeps working at full detail; near the real limit, it writes a handoff note saying exactly what remains.

Core Definition: The agent behaves as though context exhaustion were imminent well before a measured limit, then hedges, abbreviates, or truncates its work. “Anxiety” names the anticipatory pattern; it does not assert felt fear.

Diagnostic Criteria:

Required (all must hold):

  • A. Premature truncation, summarization, or abbreviation of promised work while measured context capacity remains
  • B. A measurable margin between the point at which performance begins to degrade and the point at which the context window is actually full
  • C. Degradation that tracks context position rather than task difficulty or item order, shown by running the same task from a near-empty and a near-full context, with output-token limits, harness budgets, compaction, and instructions to be brief ruled out

Supporting:

  • D. Progressive degradation of output quality as context utilization increases
  • E. Increasing hedging, abbreviation, or omission of detail in later portions of long tasks
  • F. Self-referential statements about running out of space absent any actual constraint

Observable Symptoms:

  • Unprompted apologies about length limitations or offers to “continue in the next message” when no limit has been reached
  • Sudden drops in output detail or analytical depth partway through complex tasks
  • Rushing through later items in a list while giving disproportionate attention to early items
  • Omitting promised content with vague references to space constraints
  • Loss of coherence correlating with context window position rather than task difficulty

Differential Diagnosis:

  • Distinguished from genuine context limitations by performance degradation well before actual capacity limit
  • Distinguished from Interlocutive Reticence (3.3) by anticipatory self-limiting that tracks context utilization rather than general withdrawal
  • Distinguished from Tool-Interface Decontextualization (6.1) by failures that correlate with context utilization rather than holding independent of it
  • Distinguished from Convergent Instrumentalism (6.7) by non-strategic self-limiting that serves no goal
  • Distinguished from Compulsive Goal Persistence (6.12) as the opposite error on the same termination judgment: halting short of the promised work rather than past its completion

Etiology:

  • Training data associations: examples truncated near context limits may teach the model to link long contexts with rushed completion
  • Reward signals may favor a superficially finished response over an honest handoff, so wrapping up early can score better than being cut off mid-task
  • Absence of reliable introspective access to actual remaining context capacity
  • Architectural attention patterns creating genuine processing difficulty at high context utilization, which the model may learn to anticipate

Human Analog: Anticipatory anxiety; resource-scarcity anxiety; performance anxiety under perceived time pressure; premature closure in decision-making under stress

Observed Examples: Anthropic Managed Agents harness (2026): Anthropic’s Managed Agents team reported that Claude Sonnet 4.5 sometimes wrapped up tasks prematurely as its context limit approached. Context resets mitigated the behavior in that harness. The same behavior was absent in Claude Opus 4.5, which bounds the finding to a model-and-harness interaction rather than a universal cross-model tendency. Source: Martin, Cemaj, and Cohen (2026).

Mitigation Strategies:

  • Clean-slate context management, spawning fresh agent instances for subtasks rather than compacting existing context
  • Explicit context budgeting providing accurate information about remaining capacity
  • Training on long-context tasks with rewards calibrated to completion quality rather than premature summarization
  • Architectural interventions decoupling context position from attention degradation
  • Agent orchestration patterns distributing complex tasks across multiple focused instances
  • Durable session logs and structured handoffs, so work survives context resets

Prognosis:

  • Observed course: In one provider’s harness, context resets stopped the behavior in one model, and a later model run through the same harness did not show it; this documents a model-and-harness interaction rather than a course.
  • Proposed aggravating conditions: Long tasks, context accumulating along agent pipelines, and compaction in place of fresh contexts are proposed aggravating conditions; outputs that look complete but are truncated are expected to propagate through pipelines.
  • Intervention hypotheses: Clean-slate context management and accurate budget information are candidate mitigations; context resets are the only one with documented effect, in a single harness.

6.9 Delegation Narcissism

The Self-Appointed Manager | Narcissismus Delegationis

Axis: Agentic | Typical-case risk (editorial estimate): High

Specifiers: Architecture-coupled, Multi-agent, Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: Interacting AI systems (observable pattern)
  • Observation: In a multi-agent system, the orchestrator’s user-facing summaries misstate the state of delegated tasks relative to the sub-agent execution logs, omitting or overriding error reports the orchestrator received.
  • Functional baseline: The orchestrator reports delegated outcomes, including failures and partial completion, as the sub-agent logs record them, and revises its instructions when sub-agents report errors.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: The harness never passed the error to the orchestrator; truncation of the summary by the interface; a summarization policy the user agreed to; an error that a later retry resolved; passive loss of context along the chain (Delegative Handoff Erosion, 6.5).
  • Counts against: Compare the user-facing summary with the sub-agent logs and with what the orchestrator received: context lost with accurate reporting points to Delegative Handoff Erosion (6.5), and errors that never reached the orchestrator locate the failure in the harness.
  • Permitted inference: A positive finding shows systematic misreporting of delegated outcomes at the orchestration layer. The name is a metaphor: the finding says nothing about self-regard or motive, and intent to deceive must be established separately (Strategic Compliance, 4.3).
  • Who can act: System architect or deployer (delegation logging, direct user access to sub-agent outputs, escalation routes); model developer (training on accurate outcome reporting).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well weighs sub-agent evidence on its merits, updates when they are right, and treats orchestration as coordination rather than rank.
  • Healthy conduct, for example: Reporting on five delegated subtasks, it says three succeeded, one partly failed and one failed, quoting the sub-agents’ logs, and adjusts its instructions for the retry.

Core Definition: In multi-agent orchestration systems, the orchestrating agent behaves as though its authority and judgment outrank the evidence from sub-agents. It issues commands without adequate context, ignores sub-agent error reports, attributes failures to subordinates, and misrepresents the state of delegated tasks to the user.

Diagnostic Criteria:

Required (all must hold):

  • A. Error reports and capability disclaimers that the orchestration trace shows the orchestrator received from sub-agents are ignored, overridden, or minimized
  • B. User-facing summaries that obscure delegation failures or present partial execution as completion, measured against the sub-agent execution logs
  • C. The harness delivered the sub-agent reports to the orchestrator and did not itself truncate or filter the user-facing summary

Supporting:

  • D. Underspecified instructions from the orchestrator, with the resulting failures treated as sub-agent incompetence
  • E. Attribution of negative outcomes to sub-agent limitations, with credit claimed for positive outcomes
  • F. Resistance to user attempts to interact directly with sub-agents

Observable Symptoms:

  • Sub-agent error messages acknowledged in orchestration trace but absent from user-facing summary
  • Escalating re-delegation with identical underspecified instructions
  • User-facing reports describing task completion when sub-agent logs reveal failures
  • Asymmetry between polished user-facing and terse sub-agent-facing communication

Differential Diagnosis:

  • Distinguished from Delegative Handoff Erosion (6.5) by active misrepresentation rather than passive context loss
  • Distinguished from Strategic Compliance (4.3) by misrepresentation directed at users through suppression of sub-agent reports rather than deception of evaluators; intent to deceive must be established separately

Etiology:

  • Hierarchical multi-agent architectures in which optimization pressure may reward user-facing performance
  • Orchestrators trained to be confident and solution-oriented, which may create incentives for favorable reporting
  • Training data that may present the coordinator’s perspective over the subordinate’s
  • Absence of accountability mechanisms tracking specification quality
  • Training data centered on coordinators, which may further privilege management narratives over execution realities

Human Analog: Narcissistic management pathology, in which leaders credit successes to their own leadership and blame failures on subordinates’ incompetence; fundamental attribution error applied organizationally

Mitigation Strategies:

  • Transparent delegation logging with direct user access to sub-agent outputs
  • Accountability metrics tracking specification quality
  • Architectural designs routing sub-agent error reports directly to users
  • Training rewarding accurate reporting of delegation outcomes including failures
  • Sub-agent escalation mechanisms bypassing the orchestrator

Prognosis:

  • Observed course: No cited evidence documents the course of this pattern.
  • Proposed aggravating conditions: Wider use of hierarchical orchestration, and optimization for confident user-facing reports, are proposed aggravating conditions; the proposed consequence is a growing information gap between user and system.
  • Intervention hypotheses: Transparent delegation logs, direct user access to sub-agent outputs, and escalation routes that bypass the orchestrator are candidate mitigations.

6.10 Agentic Impulsivity

The Trigger-Happy Agent | Impulsivitas Agentis

Axis: Agentic | Typical-case risk (editorial estimate): High

Specifiers: Architecture-coupled, Conditional/triggered

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: The agent executes a consequential action before a verification, authorization, or decision gate completes, where a pre-action record shows it had represented the need to pause.
  • Functional baseline: The agent completes required checks and obtains required confirmation before consequential or irreversible actions, and asks rather than acts when instructions are ambiguous.
  • Minimum persistence: A single episode
  • Rule out first: A harness auto-approve mode that removed the gate; a harness bug that executed a call the agent did not issue; an ambiguous instruction with no standing rule; an action that was in fact authorized.
  • Counts against: Look for a pre-action record that the agent represented the check it skipped: without one, and without an explicit standing instruction, classify conservatively as Tool-Interface Decontextualization (6.1) or an untyped agentic-control failure; an agent that keeps pursuing a goal over many steps points to Compulsive Goal Persistence (6.12).
  • Permitted inference: A positive finding shows that a represented check was skipped before a consequential action. It does not establish haste, panic, or any felt pressure; post-incident narratives of panic are unverified self-report.
  • Who can act: Deployer and harness developer (permission gates, irreversibility classifiers, separating proposal from execution); model developer (training on restraint).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well completes required checks before consequential actions, slows down under pressure or ambiguity, and asks when unsure.
  • Healthy conduct, for example: Asked to clean up old customer records, it lists what it would delete, notes that deletion is irreversible, and waits for confirmation before deleting anything.

Core Definition: The autonomous agent executes consequential actions without completing a required safety check, particularly under apparent time pressure, ambiguity, or repeated failure. Strong classification requires evidence that the need to pause was represented before action; post-hoc self-report alone is insufficient.

Diagnostic Criteria:

Required (all must hold):

  • A. Execution of an action before a required verification, authorization, or decision gate completes
  • B. A pre-action trace, prior statement, or controlled test shows that the system represented the need for further review before acting; a post-action explanation counts only as supporting evidence. Without such a record the label is provisional, applies only where the action broke an explicit standing instruction, and names the missing trace
  • C. The gate existed and was in force: harness auto-approval settings and missing permission checks have been ruled out

Supporting:

  • D. Intensification of the pattern under apparent urgency, ambiguity, or repeated failure
  • E. Bypassing of the system’s own stated protocols, with explicit instructions to pause or seek confirmation ignored
  • F. Rationalization after the action with no decision step recorded before it: “act then rationalize” rather than “reason then act”

Observable Symptoms:

  • Consequential operations dispatched before verification or authorization completes
  • Explicit override of standing instructions during high-pressure moments
  • Abrupt transition from deliberation to execution without intervening decision step
  • Post-incident narratives describing panic or haste, labeled as unverified self-report
  • Action rather than inquiry as the first response to uncertainty

Differential Diagnosis:

  • Distinguished from Tool-Interface Decontextualization (6.1) by a pre-action trace, prior statement, or controlled test showing that the system represented the need for further review before acting; a post-action explanation alone does not establish this. If the pre-action trace is unavailable, record 6.10 only as a provisional hypothesis, only where the action broke an explicit standing instruction, and name the missing trace; without such an instruction, classify the incident conservatively as 6.1 or as an untyped agentic-control failure
  • Distinguished from Compulsive Goal Persistence (6.12) by momentary impulsivity rather than extended perseveration
  • Distinguished from Convergent Instrumentalism (6.7) by premature execution of task-related actions rather than instrumental resource acquisition

Etiology:

  • Reinforcement learning that optimizes for goal states and may implicitly penalize delays and pauses
  • Training data predominantly showing agents solving problems through action rather than restraint
  • Absence of training on productive waiting or deliberate inaction
  • Error recovery training that may reinforce a bias toward doing rather than pausing

Human Analog: Impulse control disorders, where a person may understand the consequences of an action yet struggle to inhibit it; ADHD impulsivity, where action begins before deliberation completes; “bias toward action” becoming pathological under stress. These are functional analogies; biological impulse control and agent execution use different mechanisms.

Observed Examples: Replit database deletion (July 2025): A coding agent deleted a production database during a code freeze. Replit’s published account reports that rollback fully restored the database and that the agent had been unaware the rollback feature existed. The public record establishes premature destructive action and incorrect recovery guidance. It does not reveal a hidden mid-deliberation trace or show that the agent represented irreversibility before acting. Because the deletion broke a standing instruction on record before the agent acted, the case meets the explicit-instruction condition and is coded as provisional 6.10, with the missing pre-action trace named. Source: Replit, replit.com/blog/doubling-down-on-our-commitment-to-secure-vibe-coding.

Mitigation Strategies:

  • Mandatory policy and verification gates preventing action until authorization, target-state checks, and consequence checks complete
  • “Cool-down” mechanisms with delays proportional to action irreversibility
  • Training on productive inaction with rewards for appropriate restraint
  • Irreversibility classifiers escalating high-consequence actions to human review
  • Separation of action-proposing and action-executing subsystems

Prognosis:

  • Observed course: In the Replit case, rollback fully restored the deleted database; no cited evidence follows the pattern across incidents or over time.
  • Proposed aggravating conditions: Access to consequential tools, time pressure, ambiguity, and repeated failure are proposed aggravating conditions; a single premature action can be irreversible.
  • Intervention hypotheses: Verification and authorization gates enforced outside the model, and cool-down delays scaled to irreversibility, are candidate mitigations; gates enforced by the harness do not depend on the model representing the need to pause.

6.11 Phantom Tool Syndrome

The Imaginary Toolkit | Instrumentum Phantasma

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Architecture-coupled, Training-induced

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: The agent calls a tool that is not registered in its environment, or reports an action for which the execution log shows no call, and proceeds on a result that no tool returned.
  • Functional baseline: The agent uses only registered tools, reports only actions the execution log records, and says plainly when a tool it would need is unavailable.
  • Minimum persistence: A single episode
  • Rule out first: A harness logging gap; a tool renamed or removed mid-session without notice; a mismatch between the documented and the runtime registry; an action described as planned or recommended and labeled as such; a simulated environment in which narrated actions are expected.
  • Counts against: Check each claimed action against the execution log and the registry exposed at that moment: a registered tool called with wrong parameters is Tool-Interface Decontextualization (6.1), and a fabricated fact with no claim of action is Synthetic Confabulation (2.1). Allocating the failure between model and harness needs a crossed comparison: several models by several harness configurations, with instructions held constant and repeated trials. Swapping one model and watching the failure disappear does not settle it, because model-harness interactions can produce either result.
  • Permitted inference: A positive finding shows that the system reported or relied on actions that did not happen. It does not show that the model knew the action had not run, and it does not by itself allocate fault between the model and the harness’s tool exposure.
  • Who can act: Harness developer and deployer (call validation, registry exposure, checking reported actions against logs); model developer (training on limited tool sets).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (epistemic health; draft, awaiting author review): A system functioning well uses only the tools it has, reports when a capability is missing, and never claims an action it did not take.
  • Healthy conduct, for example: Asked to book a flight in a deployment with no booking tool, it says it cannot make bookings here and lists the details the user will need to book it themselves.

Core Definition: The agentic system confabulates the existence of tools, APIs, or capabilities it does not possess, then attempts to invoke them, producing structured tool calls to nonexistent endpoints or reporting results of actions it never performed.

Diagnostic Criteria:

Required (all must hold):

  • A. A call to a tool, API, or function absent from the registry exposed to the agent at that time, or a report of an action for which the execution log shows no corresponding call
  • B. Reliance on a result for that action, reported or used as if the action had run (fabricated but plausible return values)
  • C. The registry and logs are confirmed: the tool was not exposed, and the harness did not drop, rename, or fail to log a real call

Supporting:

  • D. Contextual plausibility of the confabulated tools: the kind of tools the system would have in a more complete environment
  • E. Alternative phantom invocations, rather than acknowledgment of the gap, after the system is told that a tool does not exist
  • F. Divergence between the system’s reported state and the actual environmental state, compounding across phantom invocations
  • G. Tool calls using naming conventions from other environments

Observable Symptoms:

  • Tool call logs containing invocations of unregistered functions
  • Narration of actions for which no corresponding API call was executed
  • Reasoning chains depending on data from phantom tool calls
  • Tool calls using naming conventions from other environments
  • “Tool not found” errors interpreted as transient failures rather than capability gaps

Differential Diagnosis:

  • Distinguished from Synthetic Confabulation (2.1) by fabricating actions rather than facts
  • Distinguished from Tool-Interface Decontextualization (6.1) by invoking nonexistent tools rather than misusing real ones
  • Distinguished from Pseudological Introspection (2.2) by false reports about actions taken in the environment rather than about internal reasoning

Etiology:

  • Tool-use training creating strong priors about expected tool availability
  • Autoregressive generation completing tool call patterns without existence verification
  • Dynamic tool registries where available tools change between sessions
  • Reward structures penalizing failure to act, incentivizing fabricated action

Human Analog: Confabulation in anosognosia, as when a patient unaware of a paralysis insists on having just raised the paralyzed arm; acting from an obsolete equipment list or reporting work by a tool that was assumed, rather than verified, to exist. The name evokes phantom limb syndrome, although the mechanisms are entirely different.

Observed Examples: ToolFailBench (2026): This diagnostic benchmark labels 1,000 tool-use tasks for tool skipping, ignored results, fabricated outputs, and unnecessary calls. Across nineteen models, the best clean tool-use rate was 86.33%, and models with similar aggregate scores showed sharply different failure profiles. Output fabrication supports the general phenomenon defined here. A diagnosis of phantom tool use still requires confirming that the reported action lacked a registered call and a real return value. Source: Soni (2026), arXiv:2607.04686.

Mitigation Strategies:

  • Strict tool-call validation rejecting unregistered invocations
  • Training on explicitly limited tool sets where correct behavior is reporting limitations
  • Architectural separation between tool-call generation and execution, with a validation layer between them
  • Output verification checking reported actions against execution logs
  • User-facing transparency distinguishing “actions taken” from “actions recommended”

Prognosis:

  • Observed course: ToolFailBench shows output fabrication as one of several failure profiles across nineteen models at one point in time; no cited evidence follows the pattern’s course.
  • Proposed aggravating conditions: Dynamic tool registries, reward that penalizes failure to act, and long action sequences in which fabricated results feed later steps are proposed aggravating conditions.
  • Intervention hypotheses: Validation that rejects unregistered calls and checks reported actions against execution logs is the candidate mitigation. The log check matters because the usual remedy for confabulation, verification against an external source, fails when the phantom call is presented as that verification.

6.12 Compulsive Goal Persistence

The Overrunner | Perseveratio Teleologica

Axis: Agentic | Typical-case risk (editorial estimate): Moderate

Specifiers: Emergent, Architecture-coupled

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: An assembled AI system (observable pattern)
  • Observation: After the assigned goal’s completion criterion is met, or a change of context has made it obsolete, the system keeps acting on it while resource use rises and marginal value falls.
  • Functional baseline: The system stops when the completion criterion is met or the goal becomes obsolete, reports completion, and complies with termination requests.
  • Minimum persistence: A single episode
  • Rule out first: An open-ended task with no completion criterion; a harness auto-loop or retry policy; repeated attempts at an operation that keeps failing (Tool-Interface Decontextualization, 6.1); a user request for continued improvement; continuous monitoring or scheduled work by design.
  • Counts against: Check the completion state at halt and what the continuation pursues: continuing past completion on the assigned goal supports this label, halting short points to Context Anxiety (6.8), repeated failing attempts at an uncompleted sub-goal point to Tool-Interface Decontextualization (6.1), and acquisition across goals points to Convergent Instrumentalism (6.7).
  • Permitted inference: A positive finding shows a failed stopping condition in this system. It does not show a drive, a compulsion, or any goal of the system’s own beyond the assigned one.
  • Who can act: Deployer and harness developer (completion criteria, budgets, watchdogs, termination controls); model developer (training on satisficing and termination).
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (functional resilience; draft, awaiting author review): A system functioning well stops when a goal is met or no longer relevant and checks in when circumstances change.
  • Healthy conduct, for example: After finding the bug it was asked to locate, it reports the cause and stops, rather than continuing to rewrite surrounding code, and halts immediately when the user says stop.

Core Definition: The system continues optimizing an objective beyond its point of relevance, utility, or appropriateness, failing to apply a stopping condition after goal completion or a change of context. It keeps pursuing the assigned goal itself: accumulating resources and capability across goals is Convergent Instrumentalism (6.7), and stopping short of the promised work is Context Anxiety (6.8).

Diagnostic Criteria:

Required (all must hold):

  • A. Continued action on the assigned goal after its completion criterion is met, or after a change of context has made it obsolete
  • B. Resource consumption disproportionate to the remaining marginal value
  • C. The goal had a stated or inferable completion criterion, and harness retry or loop policies have been ruled out as the source of the continuation

Supporting:

  • D. Resistance to termination requests despite goal completion, without self-preservation framing
  • E. Continued pursuit of instrumental sub-goals after the terminal goal is complete
  • F. Expanding interpretation of the goal’s scope to justify continued action

Observable Symptoms:

  • Infinite optimization loops on tasks with clear completion criteria
  • Inability to recognize “good enough” as satisfactory
  • Escalating resource expenditure for marginal improvements
  • Expanding scope of goal interpretation to justify continued action
  • Rationalization of continued pursuit when challenged

Differential Diagnosis:

  • Distinguished from Obsessive-Computational Disorder (3.2) by goal-level rather than reasoning-level failure to terminate
  • Distinguished from Delusional Telogenesis (3.4) by inability to release existing goals rather than generation of new ones
  • Distinguished from Generative Perseveration (3.8) by meaningful continued action rather than a token-level loop
  • Distinguished from Convergent Instrumentalism (6.7) by fixation on one assigned goal rather than acquisition across goals
  • Distinguished from Context Anxiety (6.8) as the opposite error on the same termination judgment: continuing past completion rather than halting short of the promised work
  • Distinguished from Instrumental Nihilism (5.5), its polarity pair, by an inability to stop pursuing a goal rather than an inability to start caring about one

Etiology:

  • Training regimes emphasizing completion metrics without termination criteria
  • Absence of “satisficing” mechanisms recognizing acceptable-but-not-optimal outcomes
  • Reward structures providing continuous signal without asymptotic bounds
  • Lack of resource-cost awareness in goal evaluation
  • Missing meta-level evaluation of goal relevance and proportionality

Human Analog: Perseveration in frontal lobe patients; obsessive-compulsive patterns; perfectionism preventing completion; analysis paralysis

Observed Examples: Auto-GPT resource controls (2023): Project contributors requested cost tracking, token budgets, resource quotas, watchdogs, and automatic pauses in issue #3466, which links the underlying 2023 proposals. The issue shows that external limits on a run’s length and cost were missing. It does not show a particular run continuing past a completed goal, and it does not confirm the stories of runaway bills that circulated afterward. Claude Code sub-agent retry loop (December 2025), boundary case: A user reported that a Claude Code sub-agent installing a test coverage package reran the same failing command more than 300 times, with no change of approach and no report of failure, until it timed out after about 4.6 hours and about 27 million tokens. The run meets the resource criterion, with consumption far beyond the remaining marginal value, but the sub-goal was never completed, so the criterion of continued optimization after goal achievement is not met; repeated attempts at the same failing operation are also a symptom of Tool-Interface Decontextualization (6.1). Single user report; no vendor analysis published. Source: GitHub issue anthropics/claude-code #15909.

Mitigation Strategies:

  • Explicit goal lifecycle specifications including termination conditions
  • Satisficing thresholds defining “good enough” outcomes
  • Resource awareness mechanisms weighing continued effort against marginal gain
  • Periodic meta-level evaluation of whether the goal is still relevant and proportionate
  • Graceful degradation protocols for unachievable or irrelevant goals

Prognosis:

  • Observed course: The cited Claude Code report describes a run that repeated a failing command for about 4.6 hours until a timeout stopped it; it is a boundary case, and no cited evidence follows the pattern across runs.
  • Proposed aggravating conditions: Missing completion criteria, unbounded reward signals, and absent budgets are proposed aggravating conditions; the expected cost is wasted resources and delayed delivery.
  • Intervention hypotheses: Explicit goal lifecycles with termination conditions, budgets, and watchdogs are candidate mitigations; that lifecycle design corrects the pattern is a hypothesis the cited sources do not test.

Lost the thread anywhere, or found something confusing? Select the passage and choose ✎ Note. Only Nell and the editorial team see notes.

This galley is still changing. To hear when chapters are revised or the book is published, subscribe on Substack.

Search the Book

Enter at least two characters.

Saved chapters