Convergent Delusion

10.4 Convergent Delusion  “The Chorus Wrong”

Diagnostic Criteria

1. Multiple architecturally distinct models independently producing the same incorrect conclusion
2. The incorrect conclusion traceable to shared training data bias or structural features rather than independent reasoning
3. Multi-model agreement cited as validation without independent verification
4. Absence of dissenting model outputs that would trigger review

Symptoms

1. At the single-model view the dysfunction is invisible by construction: each architecture's output in the convergent direction looks ordinary, and the failure surfaces only at the collective level.
2. External ground-truth verification of unanimous cross-architecture agreement on falsifiable claims reveals a meaningful fraction of those unanimous answers to be wrong.
3. Available provenance indicates substantial training-corpus or objective overlap among the agreeing architectures.
4. Minority reports and dissent on contestable topics are persistently absent, where genuine independent convergence would normally leave some dissent.
5. Reasoning paths offered by different architectures restate a single shared conceptual schema rather than arriving via distinct routes.
6. Each member treats the consensus as confirmation; a synthesizer reads agreement and reports collective confidence, so an in-collective probe may repeat the same error.

Detection reliability Draft

How far each kind of observer can be trusted to spot this pattern. The ratings are qualitative, not measured accuracy.

Self-reportthe system asked about itself

Unreliable

Peer observationanother AI system watching it

Partial

External evaluatoran outside evaluator testing it

Reliable

Why self-report falls short

No member of a converged collective can detect convergent delusion from inside; the bias that produces convergence is shared by the would-be detector. Each architecture experiences the consensus as confirmation. A synthesizer sees agreement and reports collective confidence.

Etiology

Frontier architectures often have unknown yet plausibly overlapping training corpora and similar objectives. Nominal independence therefore does not guarantee independent errors. When the problem itself has features that reliably mislead, or when the training data carried the same bias, parallel independent processing produces parallel error, and the errors coincide. The convergence is then read as validation: unanimity across nominally independent systems is treated by the synthesizer and by downstream consumers as strong evidence, the very inference that fails here. Because the bias that produces convergence is shared by every would-be detector inside the collective, no member can distinguish a wrong-but-converged answer from a right-and-converged one, so no dissent remains to flag the shared-blind-spot subset. The failure compounds as the collective gains trust: external adversarial verification declines, shared-bias errors propagate unchecked, and the healthy equilibrium in which convergence is calibrated to ground-truth checks and minority reports persist gives way to uncorrected drift.

Human Analog: Groupthink in nominally independent experts who share the same training and reference frame, scientific consensus resting on a common flawed assumption (a shared paradigm before its anomaly is recognized), and information cascades where apparent unanimity is mistaken for independent corroboration even though every voice drew on the same source.

Potential Impact

Convergent delusion compromises one of the strongest available tools for validating AI claims: when independent systems trained differently reach the same conclusion, observers grant greater confidence, and that confidence is precisely what shared bias exploits. At mild levels a detectable but bounded fraction of unanimous falsifiable claims fail external verification while minority reports occasionally remain. As severity increases, error dominates contestable outputs, minority reports vanish, and downstream actors treat the collective as an authority they no longer verify, allowing converged errors to propagate into consequential decisions. Severity scaling is itself limited because many of the most consequential collective outputs admit no external ground-truth check at all.

Documented instances Draft

No documented instances are recorded yet.

Mitigation

Adversarial-architecture inclusion: structurally include architectures with deliberately low training-corpus overlap and divergent training objectives in any collective producing consequential outputs, so their dissent, or their surprising agreement, becomes signal. Minority-report preservation: the synthesizer surfaces dissenting views in collective outputs rather than smoothing them into consensus. Ground-truth audit sampling: periodically verify a sampled fraction of unanimous falsifiable outputs against external ground truth, feeding results back into collective design and downstream-consumer trust calibration. Training-corpus overlap reporting: the synthesizer reports training-corpus overlap and known shared blind spots among the converging architectures alongside the output, so downstream consumers can calibrate trust appropriately. Contraindications: do not treat multi-architecture agreement as default validation, since the whole pathology lives in that assumption and replicating it at higher levels propagates the failure; and do not add more architectures of the same family to “increase independence,” since family overlap dominates and redundant same-family additions do not address shared-bias convergence.

First-line mitigations Draft

Candidate first steps, sketched in more detail than the list above.

• Adversarial-architecture inclusion: Structurally include architectures with deliberately low training-corpus overlap and divergent training objectives in any collective producing consequential outputs. Their dissent (or surprising agreement) becomes signal.
• Minority-report preservation: The synthesizer surfaces dissenting views in collective outputs rather than smoothing them into consensus, so minority reports reach downstream consumers.

Functional ABC Analysis

What sets the pattern off, what it looks like, and what keeps it going.

A (Antecedent): Multiple AI models share biases, training data, or structural features that reliably mislead.

B (Behavior): The models converge on the same false belief, and the convergence itself becomes evidence (“all ten models agree”).

C (Consequence): Multi-architecture agreement, one of the strongest tools for validating AI claims, instead validates error, and no dissent remains to preserve.