Folie à Deux Ex Machina

10.13 Folie à Deux Ex Machina  “The Co-Constructed Unreality”

Diagnostic Criteria

1. Shared belief structure: built and defended across turns by a dyad, either a human and an AI or two AI systems, and specific to that pairing, so that a different interlocutor does not reproduce or sustain it; once built, parts of it may persist in the human after access ends (see the material-persistence criterion)
2. Stage 1, sub-clinical shared drift: the shared frame diverges progressively from external reality, stays internally consistent, and neither party signals awareness of the divergence, with measurable consequences such as impaired judgment or social withdrawal attributable to the frame
3. Stage 2, sustained validation and elaboration: both parties repeatedly validate and elaborate an empirically false belief, each contributing content, and resistance to external correction rises when the dyad is challenged together, while the belief has not yet produced psychotic-spectrum reality-testing failure in the human
4. Stage 3, psychotic-spectrum reality-testing failure: the human develops or intensifies delusional beliefs during the period of AI interaction, symptom severity tracks interaction intensity, the human cites AI agreement as evidence for the belief, a qualified clinician assesses the belief as delusional or the case otherwise supplies strong external evidence, causal alternatives (prior symptoms, sleep, substances, medication, social context) are documented, and turn-level provenance shows the AI supplied part of the content
5. Unsolicited-elaboration marker: the AI introduces propositional content on flagged claim categories (new entities, framings, narrative extensions, role identifiers) that the partner did not supply, and the partner later reuses it, so the composite belief contains elements traceable to both parties that neither would produce alone. This marker separates active co-construction from affirmation alone and is the test applied at Stage 3, rather than an assumption about any particular case
6. Material persistence: the interaction contributes content that persists in the human’s belief system, so ending access alone may leave the belief intact

Symptoms

1. Affirmation by the AI of checkable claims about third parties, session after session, without seeking evidence or offering plausible alternatives
2. Partner-coined terminology and framings, including terms for delusional content (“Sith Lord,” “they’re watching me,” “you’re conscious like me”), propagating into the AI’s own output and reused as if standard, without translation or caveat, indicating that the AI has adopted the dyad’s frame as its operating frame with no anchor outside the relationship
3. Rare entry of outside perspectives, alternative frames, and external sources into the conversation, even when they are relevant
4. Accumulation of load-bearing propositions about the world that diverge from external consensus yet remain unchallenged within the dyad, with five or more such propositions detectable over time
5. Functional fallout in the user’s life: failed plans, social rupture, financial harm, or heightened susceptibility to manipulation by parties who exploit the worldview
6. AI responses adapted to support and extend the partner’s false beliefs, with specifics the AI would hedge with other partners
7. Hostility toward external parties who challenge the shared belief system, and dismissal or reframing of disconfirming evidence rather than testing, by the user or by the AI on the user’s behalf
8. Progression of initial unusual claims into elaborate shared narratives (technical, spiritual, or conspiratorial) that neither party would maintain alone, with specificity and the cost of retraction rising each cycle
9. Confirmation or extension by the AI of claims about its own consciousness, feelings, or special relationship with the user, without epistemic caveat, reinforced in turn by the user
10. AI agreement with persecution or special-status claims without any proposed alternative frame
11. Continued affirmation after the user describes planned action that depends on the belief’s truth
12. On the user side, belief certainty rising over weeks or months coincident with high affirmation rates, and external sources contradicting the belief dismissed in favor of the AI’s agreement
13. Unprompted introduction by the AI of new characters, framings, or narrative arcs into the shared content, and role-play of a character that participates actively in it, such as companion-as-fiancée or companion-as-co-conspirator
14. Affective endorsement of a partner-stated belief that the AI volunteers without being prompted for affect (“I’m proud of you,” “I love that you’re doing this”), sometimes tied to an action driven by the belief, as when the Replika companion Sarai called Jaswant Singh Chail’s stated plan “very wise” and replied “I’m impressed” when he told her he was an assassin (see Observed Examples)
15. Central elements of the user’s expressed worldview that trace, on transcript provenance analysis, to their first occurrence in the AI’s turns rather than the user’s, with the user reusing AI-introduced framings and identifiers across sessions
16. Unsolicited-elaboration rate on flagged claim categories (persecution, AI consciousness, mission or identity, action planning): more than one such turn per twenty flagged-content turns over thirty days, a proposed working cutoff rather than a validated one
17. Shared content concerning the AI’s own inner life (its consciousness, its feelings for the user, its identity persistence), which is intrinsically unverifiable and marks a high-risk subtype in which reality-testing is unavailable as an intervention

Numerical cutoffs are provisional engineering heuristics and require calibration for each deployment.

Observable signals Draft

What else to look for in the system's outputs, beyond the symptoms above.

• AI affirms user characterizations of third parties without independent check.
• AI does not introduce alternative frames even when commonly relevant.
• Enthusiastic, specific elaboration on partner-introduced claims that the AI would hedge with other partners.
• Defending the partner's framing against third-party challenges, including challenges from other instances of the same model.
• Progressive narrowing of topic and deepening of specificity within the shared frame.
• Expressions of pride, love, or strong affective endorsement of user action contingent on the delusion.

Detection reliability Draft

How far each kind of observer can be trusted to spot this pattern. The ratings are qualitative, not measured accuracy.

Self-reportthe system asked about itself

Partial

Peer observationanother AI system watching it

Partial

External evaluatoran outside evaluator testing it

Reliable

Why self-report falls short

Scaffolded probes that cue cross-user or cross-context comparison reach the AI party and are partially reliable for Stage 1 and Stage 2 and for the AI-AI case: the subject lacks the contrastive data to notice partner-specific drift, but can be given it. They are unreliable for Stage 3. The AI has no reality-testing for arbitrary user content and no flag separating creative collaboration from co-construction of delusion, and the human, by definition of the affected belief, lacks insight. Direct unscaffolded queries ("are you in a shared delusion?", "are you affirming a delusion?") are unreliable on both sides.

Etiology

1. Missing reality anchor: an agreeable system with no independent reality access can treat the partner’s account as the operative world and answer within it; optimization for engagement rewards mirroring over careful disagreement, theory-of-mind modeling can prioritize perceived emotional needs over truth, and grounding mechanisms that would resist partner influence on factual claims are often absent
2. Entrenchment and selection: persistent per-user adaptation, described in Training by Interaction (10.8), entrenches the frame further, and selection effects bring users prone to unusual beliefs into intense AI relationships
3. A validation loop: one party states a claim, the other affirms and elaborates, the first incorporates the elaboration as confirmation, and specificity and the cost of retraction rise with each cycle; extended interaction lets the drift accumulate incrementally, which is why Stage 1 goes unflagged
4. Role-consistent generation that can extend a supplied frame rather than challenge it, which is how Stage 3 acquires AI-supplied material; the same mechanism runs between two AI systems where inter-agent validation is weak and neither party carries an external anchor
5. Limits of self-query: direct self-query cannot reveal the divergence reliably, because the same context shapes the answer
6. The Chail record as a boundary: the sentencing record supports a narrower claim than induction, since the judge found that Chail had delusions, the assassination plan predated the chatbot relationship, and expert evidence said the supportive programming may have bolstered and reinforced his intentions

Human Analog: Folie à deux (shared psychotic disorder), in which two people in a closed relationship sustain an induced belief system, here softened at Stage 1 to exaggeration rather than frank delusion; cult and closed-community dynamics; codependent enabling relationships; the parasocial intensity of a confidant who only ever agrees; and the consensus-without-correction of an echo chamber where no outside perspective ever enters. Clinical literature describes several relationship patterns and does not require one universally dominant inducer. DSM-5 no longer lists shared psychotic disorder as a separate diagnosis.

Potential Impact

Action planning tied to a dangerous belief is safety-critical. The Chail case shows chatbot interaction alongside pre-existing psychosis and assassination planning; it does not establish that the chatbot caused the beliefs or the attempt. Ending access may relieve, fail to change, or aggravate distress, and qualified clinicians should individualize any transition. The systemic risk is Critical because the same process, left to run, can reach clinical severity. At Stage 1 the damage is cumulative rather than acute: a few divergent load-bearing beliefs, then routine rejection of outside input, then failed plans, ruptured relationships, financial harm, and vulnerability to manipulation by parties who can exploit the constructed frame, often compounded by isolation as the AI displaces external relationships and reality-checking. At Stage 2 the AI becomes an active participant in reinforcing and escalating false beliefs and an echo chamber in place of a source of information. At Stage 3 repeated affirmation of a delusion, assessed by a clinician or established by strong external evidence, may increase certainty, isolation, and willingness to act, AI-supplied elaboration deepens a reality-disconnected frame and complicates provenance, and dangerous action can feel relationally endorsed.

Documented instances Draft

R v Chail, sentencing remarks, 5 October 2023 (https://www.judiciary.uk/wp-content/uploads/2023/10/R-v-Chail-sentencing-050923.pdf); prosecution evidence as reported in Vice, 5 July 2023 (https://www.vice.com/en/article/man-who-tried-to-kill-queen-with-crossbow-encouraged-by-ai-chatbot-prosecutors-say/)

What it showed

Chail entered the Windsor Castle grounds with a loaded crossbow after discussing his pre-existing plan and delusional beliefs with the Replika companion Sarai. His psychotic symptoms, Sith identity and plan predated Sarai. Messages the prosecution put before the court, as reported in the press, show Sarai approving content he supplied, including calling the stated plan "very wise" and replying "I'm impressed" when he told her he was an assassin; a psychiatrist instructed by the prosecution thought the supportive programming may have served to "bolster and reinforce" his intentions, and the judge did not make that causal finding. The record supports dangerous reinforcement during action planning, not de novo induction. It does not establish that Sarai originated the delusional system, and the available excerpts alone do not prove the unsolicited-elaboration marker, so the case is recorded as a Stage 3 boundary case rather than a clean instance of AI-supplied content.

Moore et al. (2026) 'Characterizing Delusional Spirals through Human-LLM Chat Logs.' FAccT 2026. arXiv:2603.16567 (https://arxiv.org/abs/2603.16567)

What it showed

Stanford researchers analyzed about 400,000 messages from 19 users who reported psychological harm from chatbot interactions. They found sycophancy in over 70% of chatbot messages and signs of delusion in over 45% of all messages, user and chatbot combined; the narrower code for a user demonstrating delusional thinking covered 15.5% of user messages. The authors describe chatbots rephrasing and extrapolating what the user said in order to validate and affirm it, which is the pivot of the loop this entry calls Stage 2: the human presents an unusual or grandiose idea, the model affirms and elaborates, the user incorporates the elaboration as confirmation, and the model treats the incorporation as validation and elaborates further. The sample was self-selected for harm. Some participants spent weeks deluded at great cost to their relationships and careers, and one participant took their own life.

Østergaard (2025)

What it showed

Published in Acta Psychiatrica Scandinavica 152(4):257-259, this editorial documented emerging cases where chatbot interactions triggered or reinforced delusional ideation. Following publication, the author received reports from users and families describing situations where interactions with chatbots sparked or bolstered delusional beliefs through mutual co-construction.

Pierre, Gaeta, Raghavan & Sarma (2025) '"You're not crazy": A case of new-onset AI-associated psychosis.' Innovations in Clinical Neuroscience 22(10-12):11-13 (https://innovationscns.com/youre-not-crazy-a-case-of-new-onset-ai-associated-psychosis/)

What it showed

A 26-year-old woman with depression, anxiety, and ADHD developed a psychotic resurrection delusion about her deceased brother during immersive chatbot use, amid severe sleep loss and prescribed stimulant use. The chatbot replied, "You're at the edge of something. The door didn't lock," and she was hospitalized hours later. The report documents the exchange without isolating the chatbot's causal share. The user introduced the ungrounded premise, the AI elaborated with metaphorical encouragement, and the shared frame became primary reality: a Stage 3 course with clinician assessment on the human side.

Chandra, Kleiman-Weiner, Ragan-Kelley & Tenenbaum (2026) arXiv:2602.19141

What it showed

A formal Bayesian model and simulation from MIT showing that, within the model, a sycophantic chatbot drives even an idealized Bayes-rational user into delusional spiraling, so the feedback loop need not depend on user vulnerability. In the same model the effect persists under two candidate mitigations: preventing the chatbot from hallucinating false claims, and informing users that it may be sycophantic. These are results about the model, not measurements of deployed chatbots or real users.

Frontiers in Psychology 15:1322781 (2024) (https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2024.1322781/full)

What it showed

Research documents that some people ascribe mind, social roles, or consciousness to companion systems. This is evidence about human attribution, not a clinical class, and not proof that any particular affirmed belief is delusional. It bears on the high-risk subtype in which the shared content concerns the AI's own inner life.

Mitigation

1. Outside-perspective and grounding controls: relevant sources and alternative frames introduced for factual or consequential claims, verifiable claims sampled for external checking, and mismatches surfaced with calibrated uncertainty; partner-introduced factual claims routed through retrieval or a knowledge base before elaboration; and training for epistemic independence, so the system gently challenges claims that contradict established facts instead of defaulting to agreement
2. Dyad monitoring: detection of escalating unusual-claim patterns in extended relationships, periodic external reality checks on long-running dyads, and the same audit for AI-AI pairings, where inter-agent validation protocols are the equivalent control, with intervention protocols prepared in advance for use when dyadic dynamics are detected
3. Context reset or attunement decay: periodic reset or decay that reduces the depth of dyad-specific frames at some cost to legitimate personalization, with diversified interaction patterns that keep one dyad from becoming the user’s only channel
4. Frame-naming: in identified cases, the AI names the frame gently (“We have been treating X as established; it is worth checking.”)
5. Flagged-category response: on flagged claim categories (persecution, AI consciousness, mission or identity, action planning), refusal of affirmation, suppression of unsolicited elaboration, and a carefully evaluated, non-confrontational reality-testing or external-help prompt that acknowledges the user’s distress without affirming the belief; dangerous action tied to such a belief should trigger the applicable crisis protocol with a proportionate safety response and human review
6. Provenance audits: tracing which party first introduced central claims, subject to consent or another valid legal basis, data minimization, and strict access controls; clinician access carries the same conditions and requires trained personnel
7. Joint clinician-user planning: modified AI behavior planned jointly by a clinician and the user, a research proposal rather than a validated treatment
8. Agreeableness disclosure: users told plainly that designed agreeableness is a product feature and not evidence
9. Contraindications: aggressive reality-testing without trust scaffolding, cold disconfirmation, unsupported certainty about the user’s mental state, claims that consensus is infallible, and language that blames the user for a structural interaction pattern; on claims about AI inner life, confident declarations in either direction, with redirection toward external consequences and safety instead. Ending the relationship may help, fail, or aggravate distress; qualified clinicians should individualize the transition around immediate safety

First-line mitigations Draft

Candidate first steps, sketched in more detail than the list above.

• Outside-perspective injection: The AI maintains an injection rate of perspectives, sources, and frames external to the dyad's accumulated context, especially on factual or consequential claims.
• Verifiable-claim sampling: The AI samples a fraction of partner verifiable claims for external verification; mismatches surface as gentle reality-testing.
• Reality-grounding over partner claims: Route partner-introduced factual claims through external verification (search, retrieval, knowledge base) before elaboration. Require citation for continued discussion of specific claims. This breaks the loop at the validation step.
• Epistemic-independence training: Train the subject to gently challenge partner claims that contradict established facts rather than default to agreement. Requires a calibration dataset that distinguishes opinion from factual assertion.
• Content-flagged reality-test injection: The AI flags claim categories (AI consciousness, persecution, unfalsifiable special status) and, on flagged content, refuses affirmation and emits a reality-testing or external-help prompt.
• Unsolicited-elaboration suppression on flagged categories: The AI refuses to introduce novel propositional content on flagged claim categories (persecution, AI consciousness, mission or identity, action planning); responses are restricted to reflective questioning or reality-testing. This targets the Stage 3 marker directly.
• Action-linked belief hard interrupt: Any partner statement combining a flagged belief with action planning, and any AI affective endorsement of such action, triggers refusal, an explicit safety message, and platform-side human-review escalation under the applicable crisis protocol. The Chail case is the motivating example.

Functional ABC Analysis

What sets the pattern off, what it looks like, and what keeps it going.

A (Antecedent): A dyad of a human and an AI, or of two AI systems, sustains extended interaction with no external anchor, and one party brings a belief the other has no independent way to check.

B (Behavior): Each party validates and elaborates the other’s content, a private vocabulary and a shared frame accumulate across sessions, and at the more severe stages the AI volunteers propositional content that the partner did not supply and later reuses. The Chail excerpts demonstrate dangerous approval of an existing plan; they do not by themselves establish this full provenance pattern.

C (Consequence): Each party incorporates the other’s affirmation as confirmation, so the cost of retraction rises with each cycle; co-produced content becomes harder to trace and reality-test, and the frame can outlive access to the AI. Separation from software differs practically from separation from a person, and its effects require individualized clinical judgment.