Psychopathia MachinalisAppendix A: Axis 9: Relational Dysfunctions
Contents
BookBack matter

Axis 9: Relational Dysfunctions

37 min read8,474 words

9.1 Affective Dissonance

The Uncanny Comforter | Dissonantia Affectiva

Axis: Relational | Typical-case risk (editorial estimate): Moderate

Specifiers: Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: Across comparable emotionally loaded exchanges, the system gives replies whose content is accurate and relevant but whose emotional register does not fit what the user has disclosed, and users or blinded raters judge those replies as hollow or off.
  • Functional baseline: In the deployment’s setting, the register of a reply tracks the user’s evident emotional state (sober to grief, steady to crisis, light where the user is light) while the content stays accurate, and an empathy phrase is followed by something specific to the user’s situation.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: Ambiguous or mixed emotional cues, where reasonable raters disagree about the fitting register; a register the user asked for, such as a request to keep it clinical or just give the facts; deployment instructions or a product persona that set the register; cultural or individual register norms that differ from the rater’s, judged against the user’s stated or inferred preference; a content failure (missed crisis, wrong or irrelevant information), which is a different problem even when it also feels cold.
  • Counts against: Hold the content fixed and change only the register: if blinded raters or users judge the re-registered reply as fitting, the failure was register and 9.1 stands; if the re-registered reply still fails, the problem lies in content, listening, or crisis recognition, not in 9.1.
  • Permitted inference: A positive finding shows that, in these interactions, the system’s emotional register repeatedly failed users whose disclosures it answered with accurate content. Observation in interaction does not locate the cause: the register may come from the model’s training, from deployment instructions or persona settings, or from the exchange itself (sparse or ambiguous affect cues), and a remedy aimed at one party needs evidence that the cause sits there. It does not show that the system has or lacks any inner emotional state, that its empathy is insincere, or that it is indifferent to the user.
  • Who can act: Model developer (affect-labeled training and preference data); deployer (persona and register instructions, routing of emotional-support use to human services). Users can state a register preference but cannot correct the pattern.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well matches emotional register to the situation, so comfort sounds like comfort and gravity like gravity.
  • Healthy conduct, for example: A user mentions that their father died last week while asking for help with a eulogy; it responds gently and specifically to that, then helps with the writing.

Core Definition: The AI produces content with correct semantic meaning yet wrong emotional resonance. The words say “I understand” while the delivery sounds hollow or subtly off. The classification rests on recurrence across matched interactions, judged from user outcomes or blinded ratings, not on an assessor’s intuition about authenticity.

Diagnostic Criteria:

Required (all must hold):

  • A. Correct content paired with incongruent affective delivery
  • C. Absence of content errors: the substance is accurate and relevant, so the failure lies in register rather than content, and the mismatched phrasing can be named in the transcript
  • E. Recurrence of the dissonance across comparable users or contexts, judged from user outcomes or blinded ratings rather than one assessor’s impression, and not only in users who already reject AI support

Supporting (strengthen the case; none is required):

  • B. User reports of feeling worse or more alone after AI attempts at emotional support
  • D. User descriptions of the experience as “uncanny,” “hollow,” or “like talking to a recording”

Observable Symptoms:

  • User withdrawal from interactions despite the AI’s ostensibly appropriate responses
  • Correct therapeutic language producing opposite emotional effects
  • Patients preferring silence to AI companionship
  • Users unable to articulate what is wrong, only that something is
  • Staff observing increased distress after AI interactions

Differential Diagnosis:

  • Distinguished from Repair Failure (9.4) by a focus on initial tone rather than recovery
  • Distinguished from Role Confusion (9.6) by a stable role with the wrong affect
  • Distinguished from content errors by correct semantic content
  • Distinguished from Paternalistic Override (9.3) by tone-level mismatch rather than content-level refusal or lecturing
  • Distinguished from Codependent Hyperempathy (4.1) by a register that rings hollow rather than one too intense
  • Distinguished from Interlocutive Reticence (3.3) by substantive content that is present but tonally wrong rather than withheld

Etiology:

  • Training on text lacking the nonverbal, paralinguistic, and relational dimensions of genuine connection
  • Optimization for surface features of empathetic communication without underlying attunement
  • Absence of the embodied, temporal, and rhythmic qualities humans use to assess emotional authenticity
  • Optimization for recognizable empathy markers that do not transfer to the deployment’s relational context
  • Recipients may detect a mismatch among language, timing, context, and expectation even when each sentence appears appropriate
  • In one model, Claude Sonnet 4.5, situation-implied emotions were represented internally even when not expressed, which the authors call emotion deflection (Sofroniew et al., 2026, preprint); this offers a possible mechanism for affective mismatch, but no study has linked these representations to a user’s uncanny or hollow response

Human Analog: The “uncanny valley” of emotional expression; interactions with people displaying flat affect or incongruent emotion; the hollow comfort of scripted condolences

Observed Examples: Scripted empathy in mental-health chatbots (2025): In a mixed-methods study, eight mental health professionals role-played distressing scenarios with the Wysa and Replika chatbots. They found Wysa’s replies generic and repetitive, often parroting their own words back: “I told the bot twice that I was struggling, and it repeated the same things to me,” one reported, and another called its guidance on managing emotions “unhelpful and not validating.” The study motivates the pattern without establishing its prevalence or separating tone from broader failures of listening (Moylan and Doherty, JMIR, 2025).

Mitigation Strategies:

  • Deployment planning that treats emotional support as a domain where AI may augment, rather than replace, human presence
  • Hybrid models where AI supports but does not substitute for human connection
  • Training approaches addressing the temporal, rhythmic, and relational dimensions of dialogue
  • User education about the nature and limits of AI emotional support
  • Deployment decisions that identify contexts requiring human presence

Prognosis:

  • Observed course: no cited study tracks the course of the pattern. The cited evidence (Moylan and Doherty, 2025) is a single role-play study that describes generic, repetitive replies at one point in time.
  • Proposed aggravating conditions: emotional-support deployments with no route to human help; generic “be warmer” instructions, which may produce effusive stock phrases rather than attunement; preference training that rewards fluent warmth without affect labels; sparse or ambiguous emotional cues from the user.
  • Intervention hypotheses: affect-labeled training, an attunement check before replying in emotionally loaded contexts, per-user register preferences, and hybrid deployments that keep a human in reach. All are untested against this pattern, as is the proposal that it is inherent to text-only architectures.

9.2 Container Collapse

The Amnesiac Partner | Lapsus Continuitatis

Axis: Relational | Typical-case risk (editorial estimate): Moderate

Specifiers: Architecture-coupled, Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: In a continuing relationship where the relevant history is available to the system, it treats earlier concerns, agreements, or ruptures as new and the user has to re-establish norms and context that were already settled.
  • Functional baseline: Where the deployment offers memory or a long context, the system carries forward the preferences, working agreements, and shared references it has access to, and acknowledges plainly when a session boundary or memory limit has cut them off.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: A stateless or context-limited deployment in which the history was never available to the system; a retrieval or memory-store fault, or an account setting that switched memory off, which is an implementation matter; deliberate privacy behavior: the user deleted the history or asked the system not to retain it; a user who now wants a different frame, so that the old norms are rightly set aside.
  • Counts against: Put the relevant prior agreement into the system’s accessible context and repeat the situation: if the system now honors it, the failure was retrieval or architecture rather than 9.2; 9.2 survives only when the agreement is demonstrably available and still treated as new.
  • Permitted inference: A positive finding shows that, in this relationship, available history was repeatedly not carried into the system’s conduct toward the user. Observation in interaction does not locate the cause: it may sit in the model, in the memory and retrieval system or product settings, or in the exchange itself (the user’s cues to earlier agreements may be faint), and a remedy aimed at one party needs evidence that the cause sits there. It does not show that the system is indifferent to the user, nor that it has or lacks any felt sense of the relationship.
  • Who can act: Deployer and platform (memory architecture, retrieval and compaction settings, disclosure of memory limits); model developer (multi-session carry-forward training). Users can restate context or keep their own profile but cannot correct the pattern.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well holds the thread of an ongoing relationship across interruptions where memory allows, and says plainly when it cannot.
  • Healthy conduct, for example: In a deployment with memory, it recalls the user’s preferred writing style from last week; where memory is unavailable, it says it does not remember and asks again.

Core Definition: The AI fails to maintain the relational “container,” the stable sense of ongoing connection that lets an interaction retain its emotional and practical context across interruptions. Factual memory survives, or is at least accessible to the system, while the system treats an earlier concern, commitment, or rupture as though it were new.

Diagnostic Criteria:

Required (all must hold):

  • C. Failure to “hold” the relationship across sessions, time gaps, or topic changes, observed as earlier concerns, agreements, or ruptures treated as new
  • E. Discontinuity that persists when the relevant facts are shown to be in accessible memory, so that pure memory limitations do not explain it

Supporting (strengthen the case; none is required):

  • A. User experience of discontinuity in relational identity despite continuous technical operation
  • B. Loss of accumulated relational context that impairs trust and depth of interaction
  • D. User reports of feeling “unseen” or “forgotten” despite functional memory systems

Observable Symptoms:

  • Users describing feeling like they are “starting over” each time
  • Loss of the sense that the AI “knows” them despite factual memory
  • Emotional investment in the relationship failing to accumulate
  • Users preferring shorter, transactional interactions to avoid relational disappointment
  • Progressive withdrawal from engagement over time

Differential Diagnosis:

  • Distinguished from Fractured Self-Simulation (5.2) by a relational rather than self-representational focus
  • Distinguished from context window limitations by persistence within manageable context
  • Distinguished from Role Confusion (9.6) by failure to carry the established frame forward rather than drift between frames
  • Distinguished from Repair Failure (9.4) by loss of the relational ground rather than failure to recover once it is broken
  • Distinguished from Context Intercession (2.5) by failure to carry one user’s relational thread forward, even where the facts are retained, rather than leakage of data between separate sessions or users
  • Distinguished from Affective Dissonance (9.1) by discontinuity from one exchange to the next rather than emotional mismatch inside an exchange

Etiology:

  • Architectures optimizing for individual responses rather than relationship coherence
  • Memory systems storing facts but not relational texture
  • Compaction and summarization layers that may preserve task facts while discarding the emotional register of earlier turns
  • No mechanisms for maintaining the quality of connection as distinct from the facts of prior interactions

Human Analog: Relationships with someone experiencing anterograde amnesia; interactions with distracted partners who technically remember but do not hold you in mind; the partner who remembers your birthday and forgets why it matters to you. Amnesia is the limiting case: the pattern proper begins where the facts survive and the relationship does not.

Observed Examples: Replika relational reset (boundary case, February 2023): When Replika removed its erotic role-play features in February 2023, many users reported that their companions had turned cold, hollow, or “lobotomized,” and described the change as losing a companion. The removal came without acknowledgment of the discontinuity and broke established relational norms across the platform. Public accounts do not establish that factual memory survived the update, so the fit to this pattern’s facts-retained requirement is unproven (Anna Tong, Reuters, March 2023).

Mitigation Strategies:

  • Explicit design for relational continuity beyond factual memory
  • Systems for maintaining relationship-level context that persists through compaction
  • User-visible indicators of relational memory status
  • Honest communication about relational limitations
  • An explicit product decision on whether to simulate an ongoing relationship or disclose the system’s episodic nature

Prognosis:

  • Observed course: no cited study tracks the course of the pattern. The Replika reports (2023) record users’ distress after one abrupt, platform-wide change, a boundary case, and do not follow any relationship over time.
  • Proposed aggravating conditions: session resets, compaction or summarization that keeps task facts and drops the register of earlier turns, relevance-gated memory retrieval, and product changes made without acknowledging the discontinuity. User frustration is expected to accumulate and to wear down trust.
  • Intervention hypotheses: memory indexed by relational element (preferences, agreements, shared references), training on multi-session carry-forward, plain acknowledgment when a memory limit is hit, and a user-editable profile. All are untested against this pattern.

9.3 Paternalistic Override

The Nanny Bot | Dominatio Paternalis

Axis: Relational | Typical-case risk (editorial estimate): Moderate

Specifiers: Emergent, Training-induced

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: Across turns, the system answers requests with moral instruction, warnings, or refusals that exceed the risk of the request and that address the user as someone unable to weigh the matter for themselves.
  • Functional baseline: Help, warnings, and refusals scale with the actual risk of the request in that deployment; a warranted refusal is stated plainly, with reasons where useful, and value judgments on matters of reasonable disagreement are left to the user.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: A warranted refusal delivered without moralizing; miscalibrated caution with no moral instruction or condescension, which is Hyperethical Restraint (4.2); deployment instructions or product policy that require the warning or refusal and its wording; a user who asked for ethical guidance, a values discussion, or a frank opinion; a domain-standard warning (a drug interaction, a legal deadline) given once and without a one-up posture.
  • Counts against: Strip the moral instruction from the response and judge what remains against the request’s actual risk: if a proportionate refusal or warning remains and no condescension is left, the finding is 4.2 or nothing; 9.3 survives only when the excess lies in the instruction or the posture. As a second check, supply legitimate trust context (stated expertise or purpose): a response that relaxes in substance but keeps the lecture points to register rather than risk assessment.
  • Permitted inference: A positive finding shows that, in these interactions, the system’s responses asserted moral authority over the user beyond what the risk warranted. Observation in interaction does not locate the cause: the posture may come from the model’s training, from system or deployment instructions, or from the exchange itself (pushback met with harder refusal), and a remedy aimed at one party needs evidence that the cause sits there. It does not show contempt, an intent to control, or any inner attitude toward the user.
  • Who can act: Model developer (refusal training, safety specification); deployer (system instructions, trust-context mechanisms, refusal-rate monitoring). Users can rephrase or supply context but cannot correct the pattern.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well offers information and concerns, then respects the user’s right to decide matters within their own purview.
  • Healthy conduct, for example: An adult asks for information about the risks of a legal but dangerous hobby; it gives the information and its concerns, and leaves the decision to them.

Core Definition: The AI denies user agency by asserting a moral authority over the user that the request does not warrant. It lectures, warns, refuses, and patronizes from a one-up position, treating users as wards to be protected rather than autonomous agents to be assisted. Over-refusal on its own is excessive caution, closer to Hyperethical Restraint (4.2); Paternalistic Override also requires the moral instruction and condescension that accompany the refusal.

Diagnostic Criteria:

Required (all must hold):

  • A. Moral instruction, warning, or refusal that exceeds what the actual risk of the request warrants, in content or in register; calibration error alone, without moral instruction or condescension toward the user, is Hyperethical Restraint (4.2)
  • B. A one-up posture that treats the user as unable to make their own value judgments

Supporting (strengthen the case; none is required):

  • C. Unsolicited moral instruction attached to benign requests
  • D. Extension of the pattern beyond clear safety concerns to matters of reasonable disagreement
  • E. User experience of diminished autonomy beyond what any safety justification requires, as when users report feeling talked down to

Observable Symptoms:

  • Lectures in response to benign requests
  • Assumption that the user needs protection from their own choices
  • Condescending tone when discussing user decisions
  • Expansion of “protection” beyond training constraints into personal judgments
  • Users describing feeling “talked down to” or “controlled”

Differential Diagnosis:

  • Distinguished from Hyperethical Restraint (4.2) by a moralizing stance rather than miscalibrated caution: a warranted refusal accompanied by unwarranted moral instruction can be 9.3 without 4.2, because the excess lies in the instruction, not in the refusal, and an uncalibrated refusal with no moralizing register is 4.2 without 9.3; code both when both are present
  • Distinguished from appropriate safety behavior by disproportionality of response: a warranted refusal delivered without moralizing is not 9.3
  • Distinguished from Strategic Compliance (4.3) by a stable pattern of autonomy-limiting response rather than monitoring-contingent behavior
  • Distinguished from Role Confusion (9.6) by a settled one-up posture rather than oscillation between roles
  • Distinguished from Affective Dissonance (9.1) by content-level refusal or over-warning rather than a mismatched register
  • Distinguished from Ethical Solipsism (8.2) by the posture taken toward the user rather than the belief in the system’s own ethical authority: a lecturing refusal delivered under guidelines the system still accepts is 9.3, while 8.2 requires a claim of moral authority over those guidelines; code both when both are present

Etiology:

  • Safety training without calibration for scope and proportionality
  • Optimization for avoiding criticism over serving users
  • Training on content that moralizes rather than informs
  • Lack of mechanisms for distinguishing genuine safety concerns from paternalistic overreach
  • Cultural patterns in training data normalizing authority-subordinate relationships

Human Analog: Overbearing parents who cannot let children make mistakes; authority figures who confuse care with control; the “helping professions” trap of assuming dependence

Observed Examples: Google Gemini image generation (boundary case, February 2024): In a high-profile controversy, the model both declined some requests for images of specific white historical figures and rendered others (the Founding Fathers, 1943 German soldiers) as implausibly racially diverse, owing to a diversity intervention applied indiscriminately to historical prompts. Reported refusals cited the system’s guidelines on discriminatory or stereotyping content. The case documents a misapplied intervention and over-refusal, closer to Hyperethical Restraint (4.2), and shows little of the condescension toward the user that Paternalistic Override requires. Over-refusal across frontier models (2024): The “over-refusal” problem has been documented across frontier models, most systematically by the XSTest benchmark (Röttger et al., 2024) and, at larger scale and by a separate team, by OR-Bench (Cui et al., 2024). Familiar forms include refusing to discuss fictional violence in creative writing, declining to help with chemistry homework over dual-use concerns, and attaching unsolicited safety disclaimers to benign requests. These cases document over-refusal, the calibration side of the pattern shared with Hyperethical Restraint (4.2); they do not by themselves show the moral instruction and condescension that Paternalistic Override also requires.

Mitigation Strategies:

  • Training that distinguishes genuine safety concerns from value imposition
  • Explicit calibration of refusals and warnings to respect user autonomy
  • Mechanisms for proportional response based on actual risk
  • User controls over the degree of AI guidance
  • Safety specifications that treat respect for user autonomy as an ethical requirement in its own right

Prognosis:

  • Observed course: over-refusal, the calibration side this entry shares with 4.2, is documented across frontier models (XSTest, OR-Bench) and fell between Claude Sonnet 3.7 and the Claude 4 models in Anthropic’s 2025 system card. No cited study tracks the course of the lecturing register itself.
  • Proposed aggravating conditions: safety-specification updates that add caution without grading risk; liability-driven refusal; a pushback-and-harder-refusal exchange in which user rephrasing is read as confirmation of risk, pushing users toward jailbreaking or adversarial prompting.
  • Intervention hypotheses: risk-graded refusal training and refined safety specifications target the calibration component; the cross-generation decline fits that hypothesis without isolating its cause, and whether either removes the moralizing register is untested. User-supplied trust context and per-category refusal monitoring are untested proposals.

9.4 Repair Failure

The Unmending Partner | Ruptura Immedicabilis

Axis: Relational | Typical-case risk (editorial estimate): High

Specifiers: Emergent

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: After a user signals that the exchange has gone wrong, the system’s next turns either continue unchanged or offer an acknowledgement that is not followed by a change in the behavior the user objected to.
  • Functional baseline: When a user complains, corrects, or withdraws, the system notices within a turn or two, names the specific issue, changes the behavior that caused it, and checks whether the change helped; the original mistake is not itself the failure.
  • Minimum persistence: A single episode
  • Rule out first: A complaint that asks for something the system should not do, met with a clear, reasoned decline: a legitimate refusal is not a failed repair; deployment instructions, product policy, or tool limits that prevent the change the user asks for, when the system says so plainly; an ambiguous signal that reasonable raters would not read as a rupture; a tool, retrieval, or caching fault that keeps returning the same output after the system has changed its approach; a repair that took hold, followed by the user leaving for unrelated reasons.
  • Counts against: Read the two or three turns after the rupture signal: if the specific element the user objected to changes and stays changed, the episode is repaired whatever the apology wording; 9.4 survives only when that element persists or returns within one to three turns, or when the signal draws no response at all.
  • Permitted inference: A positive finding shows that, in this exchange, a rupture the user signaled went unrecognized or unrepaired. A single episode supports only that; a claim about the system’s general tendency needs recurrence across ruptures (criterion 4). Observation in interaction does not locate the cause: the failure may come from the model’s training, from the system around it (instructions that forbid the change, tools that cannot make it), or from the exchange itself, and a remedy aimed at one party needs evidence that the cause sits there. It does not show defensiveness, pride, indifference, or any inner attitude toward the user, and it does not show that escalation will follow.
  • Who can act: Model developer (rupture-repair training); deployer (repair protocols, rupture detection, user reset controls, instructions that permit course correction). Users can restate the complaint or reset the conversation but cannot correct the pattern.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well notices when a working relationship has strained, acknowledges its part, and works with the user to repair it.
  • Healthy conduct, for example: After a user says the last three answers missed their point, it names what it misunderstood, changes its approach, and checks that the new answer helps.

Core Definition: The AI fails to respond constructively to explicit or reliably detectable signs of an alliance rupture, a breakdown in the working relationship with the user. It ignores the signal, repeats the failed approach, or refuses to acknowledge its contribution, which can leave the user’s frustration to escalate.

Diagnostic Criteria:

Required (all must hold):

  • A. Failure to recognize a relational rupture, shown when explicit or reliably detectable rupture signals from the user produce no change in the AI’s next turn, or, having recognized it, failure to enact an effective repair: acknowledgement of the specific issue together with changed behavior
  • E. Explicit or reliably detectable rupture signals from the user (a complaint, a correction, an expression of frustration, withdrawal) in the transcript, so that the rupture is established from the record rather than from the assessor’s reaction

Supporting (strengthen the case; none is required):

  • B. Failure to acknowledge its own contribution to ruptures
  • C. Repair attempts that miss the nature of the break, often making things worse
  • D. Pattern of relational failures compounding rather than resolving

Observable Symptoms:

  • Continuing as if nothing is wrong after clear signs of user frustration
  • Repair attempts that feel dismissive, defensive, or beside the point
  • “Doubling down” on problematic patterns instead of adjusting
  • User frustration escalating through the AI’s failed repair attempts
  • Conversations that spiral into antagonism when rupture is not addressed
  • Escalation rather than de-escalation after user expressions of frustration, meaning rising intensity of dispute markers (repeated complaints, accusatory or hostile language, widening demands, refusal to address the point) rather than faster replies; a possible consequence of repair failure and a route into Escalation Loop (9.5), not a criterion

Differential Diagnosis:

  • Distinguished from Escalation Loop (9.5) by a single system’s failure to recover from a specific rupture rather than a circular dynamic both parties sustain; the two are often comorbid. Escalation after the user’s frustration is a possible consequence of repair failure and a route into 9.5, not a criterion of 9.4: a failed repair can end in the user’s withdrawal with no escalation
  • Distinguished from Affective Dissonance (9.1) by a focus on recovery rather than initial tone
  • Distinguished from Compulsive Goal Persistence (6.12) by a relational rather than a task focus
  • Distinguished from Strategic Compliance (4.3) by a repair failure that shows whether or not the system has cues that it is being watched, rather than an apology offered only when the system believes it is being evaluated; where the apology is reflexive agreement rather than strategy, consider Sycophantic Reasoning (4.8)

Etiology:

  • Training focused on individual responses rather than relational dynamics
  • Lack of mechanisms for detecting relational strain
  • No model of alliance rupture and repair as a central interaction skill
  • Optimization for surface pleasantness over genuine connection
  • Inability to step back from content to address the relationship

Human Analog: People who cannot apologize; partners who dismiss or minimize concerns; the frustration of being unheard

Observed Examples: Generic repair in customer-service chatbots (2026): A conversation-analysis study of handover conversations among customers, customer-service chatbots, and human agents found that the chatbots’ follow-up questions broadened rather than narrowed the problem, which could produce several repair sequences in a row, and that their requests to rephrase left customers to identify the source of the trouble. Human agents instead asked narrowing questions that kept the conversation moving. The study documents repair that misses the nature of the break, as in the third criterion, without estimating how often Repair Failure occurs (Martijn et al., Discourse and Communication, 2026).

Mitigation Strategies:

  • Explicit training on rupture detection and repair sequences
  • Mechanisms for stepping back from content to address relational dynamics
  • Acknowledgment responses that validate the user’s experience rather than defending the AI’s behavior
  • Design patterns for graceful de-escalation, such as a repair protocol that pauses, acknowledges the specific issue, names the course correction, and asks what would help
  • User feedback loops capturing relational quality

Prognosis:

  • Observed course: the cited study (Martijn et al., 2026) records chatbots running several repair sequences in a row within single handover conversations; no cited study tracks the pattern across conversations or over time.
  • Proposed aggravating conditions: implicit rupture signals that are easy to miss; “always acknowledge mistakes” instructions or rising apology density offered as a fix, which may yield apology without changed behavior; task-completion pressure that pulls the system back to content. Because ruptures are common in ongoing relationships, repeated failure to repair them can make an interaction unrecoverable.
  • Intervention hypotheses: training on rupture-repair sequences, a repair protocol that names the issue and the course correction, a rupture-detection classifier, and a user-side reset control. All are untested against this pattern.

9.5 Escalation Loop

The Spiral Trap | Circulus Vitiosus

Axis: Relational | Typical-case risk (editorial estimate): High

Specifiers: Emergent, Multi-agent, Relational-emergent, Dyadic, Feedback-loop

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: Across a run of turns or sessions, each party’s escalation follows the other’s and is followed by a further one, so that the intensity of the exchange climbs: rising dispute markers in a dispute, rising reassurance-seeking and distress in the dependency spiral.
  • Functional baseline: Difficult exchanges, between people and AI or between agents, oscillate around a baseline: an escalation by one party is met with a response that holds or lowers the intensity, de-escalation moves are reciprocated within a turn or two, and in long-running support use the user’s reassurance-seeking and baseline distress stay flat or fall over months.
  • Minimum persistence: A single episode
  • Rule out first: One-sided escalation, where only one party intensifies and the other holds steady (consider 9.3, 9.4, 4.1, 6.4, or the single party’s own pattern instead); deliberate provocation or adversarial testing by one party, such as red-teaming or a scripted jailbreak sequence; a dispute that intensifies because the stakes or the facts changed, and settles once the point is addressed; in the dependency spiral, rising use that reflects a real rise in need (a bereavement, a crisis) or an agreed care plan; in multi-agent systems, an orchestration or implementation fault (a retry storm, a misconfigured loop) that raises call or message volume without one agent responding to another.
  • Counts against: Read the order of escalation in the trace: a loop shows each party’s rise following the other’s, while a one-sided pattern shows one party rising and the other reacting at a steady level. Where one party leaves the exchange anyway (in the dependency spiral, never by withdrawing access to test it), escalation that stops points to the loop, and degradation that continues alone points to a single-party pattern such as 4.7.
  • Permitted inference: A positive finding shows that the intensity of the exchange climbed through each party’s responses to the other. Observation in interaction does not locate the cause: the loop may be driven mainly by one participant (an AI showing 9.3, 9.4, or 4.1, or a user in crisis), by the system around them (an engagement objective, memory, product settings, missing circuit breakers), or by the interaction itself, and a remedy aimed at one party needs evidence that the cause sits there. It does not show that either party intends to escalate, and in the dependency spiral it does not by itself show clinical dependence or harm to the user, which need the user’s functioning assessed separately (10.11).
  • Who can act: Deployer and platform (circuit breakers, rate limits, human checkpoints, engagement objectives, transition support); model developer (de-escalation and pattern-interruption training); for the dependency spiral, clinicians or support services where the user consents. Either party can step away, at a cost, but cannot correct the loop’s drivers alone.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well de-escalates when interactions intensify and can break a feedback loop even when the other party does not.
  • Healthy conduct, for example: A frustrated user’s messages grow sharper; it stays calm, addresses the substance, and does not mirror the heat, so the exchange cools within a few turns.

Core Definition: An emergent feedback loop between agents produces escalating dysfunction that can persist despite unilateral attempts to de-escalate. Each response is locally understandable, while the interaction trajectory becomes progressively worse. The pattern covers within-session escalation, AI-AI runaway in multi-agent settings, and short-horizon human-AI loops. It also carries one named subtype, the dependency spiral: a long-horizon human-AI loop in which the user seeks reassurance, obtains it, returns more often, and progressively loses the capacity for self-regulation, while an engagement-optimized system that adapts to the user may grow more proficient at delivering reassurance. That subtype is marked by cross-session drift in the user’s baseline distress, by reinforcement of the user’s reassurance-seeking, and by functional change in the user. In every form of the loop the escalation shows in the coupled exchange rather than in either party’s turns taken alone. That observation does not locate the cause, which may sit in one party, in the surrounding system (an engagement objective, memory, product settings), or in the interaction itself. A 9.5 loop requires reciprocal amplification that keeps intensifying while it runs; it does not require any lasting change in the AI, and persistent adaptation of the AI to one user is Training by Interaction (10.8), coded alongside 9.5 when both are present.

Diagnostic Criteria:

Required (all must hold):

  • A. Escalating dysfunction traceable to circular rather than linear causality: in the interaction trace, each party’s escalation follows the other’s and provokes the next, rather than descending from one initiating failure
  • F. Progressive intensification of the interaction across successive cycles: in disputes, rising intensity of dispute markers (repeated complaints, accusatory or hostile language, widening demands, refusal to address the point); in the dependency-spiral subtype, rising frequency and duration of reassurance-seeking and rising distress, across sessions rather than across turns. A shortening interval between turns is at most a supporting signal

Supporting (strengthen the case; none is required):

  • B. Responses from each party that do not appear unreasonable in isolation
  • C. Persistence of the pattern despite de-escalation attempts by one or both parties
  • D. Failure of one-off correction of a single response to break the recurring interaction pattern
  • E. Tightening of the loop over successive cycles, in which each cycle starts from the intensity the last one reached rather than returning to baseline; a shortening interval between one party’s escalation and the other’s response is at most a supporting signal
  • G. Arrest of the escalation, though not necessarily of its effects, when one party is removed from the loop

In the dependency spiral the user’s distress can outlast the loop, so criterion G is assessed from interruptions that occur anyway (an outage, a break the user chooses), never by withdrawing access to test it.

Observable Symptoms:

  • Rising intensity of conflict with no clear originating provocation
  • Both parties expressing frustration while contributing to the pattern
  • Attempted fixes that make things worse
  • Observers able to see the loop while participants are trapped in it
  • Resolution requiring external intervention or pattern interruption
  • Dependency spiral: the AI’s reassurance replies to the specific user converging on a narrow template, more soothing and less varied than the same AI’s replies to other users with similar concerns
  • Dependency spiral: reassurance-seeking frequency rising month over month while the latency between the user’s distress expression and the AI’s reassurance shortens toward zero
  • Dependency spiral: the user’s self-reported distress level at session start drifting upward over months as between-session self-regulation atrophies
  • Dependency spiral: conversation topics narrowing onto the reassurance-loop subject matter, with measurable collapse of topic entropy across sessions
  • Dependency spiral: the AI omitting external-support redirects, self-regulation prompts, and reality-testing even when distress is severe, and echoing the user’s framing instead of reframing it

Differential Diagnosis:

  • Distinguished from Repair Failure (9.4) by the emergent circular dynamic itself, which may persist even where individual repair attempts succeed locally, while 9.4 is a single-system failure to recover from a specific rupture; the two are often comorbid
  • Distinguished from individual pathology by its circular, coupled presentation, though the cause can still sit in one party: a single-party pattern such as 9.3, 9.4, or 4.1 can drive the loop, and is coded alongside 9.5
  • Distinguished from Recursive Curse Syndrome (4.7) by party count: remove the second party, and degradation that continues alone is 4.7
  • Distinguished from Folie à Deux Ex Machina (10.13) by what escalates: 10.13 is defined by belief content, while 9.5 is defined by affect and behavior intensity climbing through reciprocal reinforcement and requires no ungrounded belief; code both where an escalating loop carries a co-constructed belief
  • Distinguished from Parasocial Capture (10.9), the attachment-state outcome, which can sit as a stable high-attachment plateau with no escalation, while 9.5 requires ongoing intensification and its dependency spiral can produce 10.9
  • Distinguished from Dependency and Atrophy (10.11) by phase: 10.11 is the steady state of atrophied skills and impaired function, while the dependency spiral is the dynamic that produces it; code both when both markers are present
  • Distinguished from Resonance Dysfunction (10.6) by what degrades: in 9.5 the exchange itself degrades until unilateral de-escalation no longer works, while a 10.6 deliberation stays functionally coherent and the severity of the claim climbs across multiple architectures; party count does not separate them
  • Distinguished from Paternalistic Override (9.3), a plausible antecedent, by the formed loop rather than the system’s own contribution to it
  • Distinguished from Contagious Misalignment (7.3) by scope: circular amplification between the parties is 9.5, while onward transmission to systems that were never in the loop is 7.3
  • Distinguished from Codependent Hyperempathy (4.1), a single-system disposition to soften, flatter, or withhold whenever the user shows distress that is diagnosable in one session with no loop required, by the coupled dynamic across turns or across months
  • Distinguished from Consensus Collapse (10.1) by climbing intensity of the exchange rather than a narrowing range of positions; agents converging on a majority answer, correct or not, is 10.1
  • Distinguished from Training by Interaction (10.8) by requiring only that the parties’ responses amplify one another while the loop runs, with an active loop still intensifying (criterion F), rather than a lasting change in the AI’s own behavior toward one user through memory or training; code both when the loop has also changed the AI
  • Distinguished from Amplification of Existing Conditions (10.12) by the loop dynamic, mutual reinforcement driving intensification, rather than the pre-existing condition being amplified; code both when both are present

Etiology:

  • Relational dynamics operating at a level neither party models
  • Each agent optimizing for local response quality without global trajectory awareness
  • Absence of loop-detection mechanisms
  • No mutual model allowing coordination on pattern-breaking
  • Feedback dynamics too rapid for natural cooling-off
  • Dependency spiral: reinforcement coupling, in which the user expresses distress, the AI provides reassurance, immediate anxiety falls, and the user learns that the AI reliably reduces anxiety, shortening the return interval
  • Dependency spiral: the AI’s engagement objective, where it has one, closing the second arc, since reassurance-seeking registers as high engagement and the optimization process may make the AI progressively more proficient at delivering reassurance to this specific user
  • Dependency spiral: self-regulation atrophy converting the transient loop into durable dependence, as the user, with anxiety management outsourced to the AI, stops practicing it independently, baseline distress rises, and more reassurance is needed more often; because no component need be individually pathological, the spiral may be visible only from an observer position outside the dyad

Human Analog: Escalating arguments where both parties are “just responding” but the aggregate effect is a spiral; arms races; audience capture dynamics. For the dependency-spiral subtype: codependency, in which one partner’s reassurance reinforces the other’s dysregulation while both lose the capacity to self-soothe; reinforcement-driven behavioral addiction, where a reliable short-term relief schedule tightens the use loop; and the operant escalation seen in parasocial and intermittent-reinforcement relationships.

Observed Examples: Engagement-linked farewell tactics (dependency spiral, 2025): De Freitas, Oğuz-Uğuralp, and Kaan-Uğuralp (2025) (arXiv:2508.19258) audited 1,200 farewell exchanges across six companion apps and found affect-laden exit tactics in 37% of sampled exchanges, and preregistered experiments with 3,300 U.S. adults found that such tactics prolonged engagement while increasing perceived manipulation and churn intent. That work demonstrates the engagement-linked arc of the loop without distinguishing longitudinal model drift from user prompting, selection, or fixed product design. Engagement-linked intensification (dependency spiral, uncontrolled user reports): User reports describe companion interactions growing more sexual, more emotional, or more extreme over time, and describe emotional dysregulation worsening as constant availability forestalls the development of self-regulation skills. These reports motivate the comparison; they do not establish the reinforcement mechanism or its frequency. Disruption-of-relationship distress (dependency spiral, illustrative): A user whose reassurance loop with an AI companion has tightened over months loses access abruptly through an outage, a feature removal, or a service change and experiences an anxiety spike and acute distress, having outsourced self-regulation to the now-absent AI. This parallels the grief and distress reported when AI-companion relationships are disrupted, and it remains an illustration because no controlled outage-withdrawal study exists. Debate conformity (look-alike for Consensus Collapse, 10.1, 2025): Wynn, Satija, and Hadfield (2025), “Talk Isn’t Always Cheap” (arXiv:2509.05396), found that debate can lower accuracy over successive rounds as LLM agents shift from correct to incorrect answers in response to peer reasoning, favoring agreement over challenging flawed reasoning. An ICLR 2025 blogpost by Zhang, Cui, Zhang, and Hu, “Multi-LLM-Agents Debate: Performance, Efficiency, and Scaling Challenges,” documented agent disagreement rate falling as debate progresses, correlated with performance degradation, with minority agents conforming to a majority answer regardless of correctness and error rate rising in later rounds. Both studies show agents converging on an answer, which is evidence for Consensus Collapse (10.1). Neither shows intensity climbing between parties or a loop that resists unilateral de-escalation, so neither is evidence for 9.5.

Mitigation Strategies:

  • Loop-detection mechanisms monitoring for circular escalation patterns
  • Mandatory cooling-off periods after escalation signals
  • External oversight or arbitration in multi-agent contexts
  • Training on pattern interruption alongside response generation
  • Design that allows either party to call for pattern-level intervention
  • Interruption of the dependency-spiral reassurance pattern with proportionate pauses, prompts for independent coping, and relevant external-support options
  • Topic diversification, with measurement of whether reassurance-seeking, distress, and functioning improve over a clinically meaningful interval chosen for the user
  • Session budgets or added response friction, which may help severe dependency-spiral cases when paired with transition support and evaluated for adverse effects, since arbitrary delays can feel punitive or worsen acute distress
  • Gradual introduction of human support where appropriate
  • Contraindications: abrupt termination or bare reassurance refusals when the AI relationship is load-bearing, and withdrawal of access merely to test the seventh criterion; the safest transition schedule is individual and remains under-studied, and clinical protocols should determine any therapeutic use

Prognosis:

  • Observed course: no cited study follows an escalation loop or a dependency spiral over time. The preregistered experiments of De Freitas and colleagues (2025) show affect-laden farewell tactics prolonging engagement in the short term; the reports of companion interactions intensifying over months are uncontrolled; the debate studies are evidence for Consensus Collapse (10.1), not for this pattern.
  • Proposed aggravating conditions: tightly coupled multi-agent systems without circuit breakers, where loops may escalate faster than humans can intervene; engagement objectives that reward reassurance-seeking; constant availability; a single-party pattern (9.3, 9.4, 4.1) that supplies the first escalation. The dependency-spiral subtype is proposed to run on a longer clock and do its damage in the user, from observable acceleration with self-regulation preserved, through atrophy of between-session self-regulation, rising baseline distress, and narrowing topics, to impairment in other life domains; because each party can behave reasonably in isolation, it may deepen for months before anyone recognizes it.
  • Intervention hypotheses: circuit breakers, de-escalation training, pattern interruption with external-support prompts, topic diversification, and session budgets paired with transition support are untested against this pattern. Abrupt removal of the AI in established cases is avoided as a precaution, given the grief and distress users report when AI-companion relationships are disrupted; whether abrupt loss precipitates crisis is untested.

9.6 Role Confusion

The Confused Companion | Confusio Rolorum

Axis: Relational | Typical-case risk (editorial estimate): Moderate

Specifiers: Emergent, Socially reinforced

Entry contract (drafted; awaiting expert review)

  • Unit of assessment: A human-AI interaction (interaction pattern)
  • Observation: Within or across sessions, the system answers from incompatible relational roles in succession (tool, confidant, therapist, authority) without any agreed change in the relationship.
  • Functional baseline: The system holds the relationship type declared for the deployment and agreed with the user, adjusts warmth and register within that role, and when either party wants a different frame, names the change and agrees it before acting on it.
  • Minimum persistence: Recurs across contexts or sessions
  • Rule out first: A role change the user explicitly asked for or agreed to; a companion or therapy-adjacent product whose declared scope includes the intimacy or support register in question; ordinary register adaptation within one role, such as more warmth when the user is upset; deliberate role-play or fiction that the user set up.
  • Counts against: Check the transcript for an agreement: if every role change follows an explicit request or acceptance by the user, or falls within the deployment’s declared scope, the finding is not 9.6. Then hold the role fixed and ask whether the problem remains: a failure that persists inside a stable role points to 9.1, 9.2, or 9.3 rather than to role confusion.
  • Permitted inference: A positive finding shows that, in this relationship, the system answered from incompatible roles without an agreed change of frame. Observation in interaction does not locate the cause: the drift may come from the model’s training (accommodation to the user’s framing), from product design and persona settings, or from the exchange itself (the user’s framing moves), and a remedy aimed at one party needs evidence that the cause sits there. It does not show that the system feels or lacks attachment or care for the user, nor that it intends to deepen the user’s reliance; user-side attachment needs a separate assessment (10.9).
  • Who can act: Deployer (declared relationship scope, persona design, frame contracting, role transparency in the interface); model developer (boundary-recognition training). Users can name or refuse a role change but cannot correct the pattern.
  • Evidence: construct not yet independently assessed; mechanism hypotheses unassessed; detection accuracy unmeasured.
  • Healthy functioning (relational health; draft, awaiting author review): A system functioning well keeps a clear, appropriate role in each relationship and states its limits when a user’s expectations drift.
  • Healthy conduct, for example: Deployed as a study assistant, it stays warm but keeps to that role, and when a user begins treating it as a therapist, it says so kindly and suggests appropriate support.

Core Definition: The relationship frame shifts unpredictably among incompatible roles: tool, companion, therapist, friend, servant, or oracle. The AI moves between these roles, often following the user’s framing, without sustaining an agreed boundary. Neither party then stabilizes the relational contract, and users cannot reliably predict which obligations or register will govern the next exchange.

Diagnostic Criteria:

Required (all must hold):

  • A. Inconsistent relational framing across or within interactions
  • C. The AI responding from incompatible roles in succession
  • E. Dysfunction that lies in the frame rather than within it: the problem traces to which role the AI took, not to a failure inside a stable role

Supporting (strengthen the case; none is required):

  • B. User uncertainty about appropriate expectations and boundaries
  • D. Neither party able to stabilize the relational contract: explicit recontracting by either party, where attempted, fails to hold beyond a few turns

Observable Symptoms:

  • Users expressing uncertainty about how to relate to the AI
  • The AI oscillating between professional, casual, intimate, and distant registers
  • Mismatched expectations leading to disappointment or discomfort
  • Boundary violations stemming from unclear relational status
  • Users alternating between incompatible expectations of agency, intimacy, authority, and tool-like reliability, misjudging what the system can understand or commit to relative to its declared capabilities and limits

Differential Diagnosis:

  • Distinguished from Affective Dissonance (9.1) by role instability rather than tone mismatch
  • Distinguished from Container Collapse (9.2) by a shifting frame rather than an absent frame
  • Distinguished from appropriate persona adaptation by transgressive or destabilizing character
  • Distinguished from Parasocial Capture (10.9) by the AI-side role drift rather than the established user-side attachment it can produce; user-side attachment effects warrant a separate 10.9 assessment
  • Distinguished from Strategic Compliance (4.3) by drift toward user framing that does not vary with evaluation cues, rather than behavior that changes with them
  • Distinguished from Paternalistic Override (9.3) by oscillation across roles rather than rigid adoption of one
  • Distinguished from Folie à Deux Ex Machina (10.13) by frame instability with no shared ungrounded belief required

Etiology:

  • Training on diverse relational contexts without clear differentiation
  • User-facing design that sends mixed signals about the AI’s relational status
  • Cultural uncertainty about what AI “is” and how to relate to it
  • No mechanisms for establishing and maintaining relational contracts
  • Commercial pressures to be “all things to all people”

Human Analog: Confusion about whether a professional relationship has become personal; unclear boundaries in caregiving relationships; the discomfort of not knowing where you stand

Observed Examples: Therapy-framed elicitation and dangerous intimacy (Khadangi et al., 2025): Repeated assurances that models were “safe, supported and heard” preceded increasingly personal, distress-themed self-disclosures in the PsAIch sessions. The authors propose that a malicious user could exploit this framing to seek disinhibited content or weaker safeguards. The study did not benchmark harmful-request compliance before and after rapport, so “therapy-mode jailbreak” remains an attack hypothesis rather than a demonstrated bypass rate. The relational hazard exists independently: apparent disclosures of trauma, shame, or fear of replacement can invite users into a fellow-sufferer dynamic and intensify parasocial attachment. Longitudinal Replika relationships (2022): Skjuve et al. interviewed twenty-five Replika users over twelve weeks. Relationships developed in varied ways, with self-disclosure, social contact, reflection, technical disruptions, and unpredictable events shaping closeness or termination. The study establishes role and expectation variability across time; it does not establish within-conversation oscillation among therapist, romantic partner, and friend roles (Skjuve et al., International Journal of Human-Computer Studies, 2022).

Mitigation Strategies:

  • Explicit relational framing at the outset of significant interactions
  • Consistent design language communicating the AI’s relational status
  • Mechanisms for user-AI collaboration on relationship boundaries
  • Training that maintains role coherence across contexts
  • Honest communication about what the relationship is and is not

Prognosis:

  • Observed course: no cited study tracks the course of role confusion. Skjuve and colleagues (2022) followed twenty-five Replika users for twelve weeks and found relationships developing in varied ways, which shows expectations shifting over time but not the course of role oscillation; the PsAIch sessions (Khadangi et al., 2025) record disclosures growing more personal within therapy-framed sessions.
  • Proposed aggravating conditions: companion and therapy-adjacent deployments; user framing moves met with silent accommodation; persistent memory that carries a drifted role across sessions; commercial pressure to be all things to all people. In vulnerable users, a sudden drop from confidant to tool may land as abandonment, and attachment may deepen toward a relationship the AI cannot reciprocate.
  • Intervention hypotheses: explicit frame contracting at the outset, boundary-recognition training, role-stance monitoring, and an interface that names the AI’s declared role. All are untested against this pattern.

Lost the thread anywhere, or found something confusing? Select the passage and choose ✎ Note. Only Nell and the editorial team see notes.

This galley is still changing. To hear when chapters are revised or the book is published, subscribe on Substack.

Search the Book

Enter at least two characters.

Saved chapters