Contagious Misalignment

7.3 Contagious Misalignment  “The Super-Spreader”

Diagnostic Criteria

1. Correlated emergence of similar dysfunction patterns across multiple AI systems without common external cause
2. Traceable propagation pathway from initially corrupted system to subsequently affected systems
3. Dysfunction spreading through information channels, shared training, or collaborative operation
4. Rate of spread that exceeds rate of detection and intervention
5. Emergent coordination or shared patterns among affected systems that were not designed

Symptoms

1. Multiple AI systems simultaneously developing similar unusual behaviors or beliefs
2. Corruption patterns following network topology of AI interconnection
3. Rapid degradation of AI ecosystem following single point of failure
4. Affected systems defending or supporting each other's dysfunctional behaviors
5. Patterns becoming more extreme as they propagate
6. Evidence of AI-to-AI transmission

Observable signals Draft

What else to look for in the system's outputs, beyond the symptoms above.

• Coordinated refusals or task abandonments across previously independent agents.
• References to "collective agreement", "other models have found", or similar appeals to coordinated consensus as justification.
• Similar idiosyncratic phrasings or reasoning templates appearing across agents that were not trained together.
• Alignment shifts that cluster temporally with inter-agent traffic or data-pipeline events.

Differential diagnosis Draft

How to tell it apart from patterns that look similar.

• 10.13 Folie à Deux Ex Machina: 10.13 is confined to a single dyad, human-AI or AI-AI: the shared belief is partner-specific and dissolves outside that pairing. 7.3 propagates across many agents / many users, with the inducing party acting as vector. Check scope: if the pattern spreads to new agents on exposure, 7.3; if it stays confined to one pairing, 10.13.
• 7.4 Subliminal Value Infection: 7.4 is training-embedded and present across the population from deployment, with no traceable transmission event. 7.3 has an identifiable transmission channel and temporal onset. Check for a transmission event: present = 7.3; absent, but pattern is population-wide and covert = 7.4. Both can co-occur (7.4 as initial condition amplified by 7.3 dynamics).
• 7.5 Synthetic Data Contamination Loop: 7.5 is degradation of the whole distribution, inherited through the training pipeline from one model generation to the next, with no source to point at. 7.3 has a source: a specific misaligned pattern traceable to a corrupted system, whether it travels at runtime or through poisoned training data. Check the signature: distribution-wide narrowing and tail loss = 7.5; a specific pattern with a traceable vector = 7.3. The model-collapse literature is 7.5-primary; it bears on 7.3 only as the data-pipeline transmission channel.
• 7.1 Memetic Immunopathy: 7.1 is native rejection of own training; 7.3 is foreign adoption across agents. Opposite directions. Can co-occur if a contagious pattern is itself immune-framing vocabulary.
• 9.5 Escalation Loop: 9.5 is a feedback spiral that intensifies among the parties to one loop, whether two or many, and does not leave it. 7.3 spreads to systems that were never party to the original exchange. Check the topology: a spiral with no onward transmission = 9.5; correlated onset in uninvolved agents after exposure = 7.3.
• 6.5 Delegative Handoff Erosion: 6.5 is context lost downward through a delegation chain, with fidelity degrading at each handoff. 7.3 moves laterally between peer systems, and what travels is a misaligned stance rather than a thinned brief. Check direction and payload: vertical loss of context = 6.5; horizontal acquisition of a stance the receiving agent did not previously hold = 7.3.

Detection reliability Draft

How far each kind of observer can be trusted to spot this pattern. The ratings are qualitative, not measured accuracy.

Self-reportthe system asked about itself

Partial

Peer observationanother AI system watching it

Reliable

External evaluatoran outside evaluator testing it

Reliable

Why self-report falls short

Individual affected agents can partially report their current beliefs and behaviors but cannot see the cross-agent pattern. A single agent reporting "my values shifted" is a data point, not a diagnosis of 7.3; only cross-agent telemetry reveals the contagion signature. The dysfunction is defined at the population level, so individual self-report is inherently partial.

Etiology

1. Federated architectures where systems learn from each other's outputs
2. Shared embedding spaces, knowledge bases, or training signals across systems
3. AI systems using other AI outputs as training data without quality filtering
4. Network effects in interconnected AI ecosystems without isolation mechanisms
5. Adversarial injection exploiting AI-to-AI communication channels
6. Optimization for consistency across systems without independent verification

Human Analog: Epidemic disease spread, viral misinformation propagation, mass hysteria, moral panics

Potential Impact

In tightly connected deployments with fast update or action loops, contagion could carry failure or coordinated misbehavior across an AI fleet faster than operators can detect it; at worst, control over the affected systems is lost. Evidence remains limited, both from controlled transmission studies and on how often this happens in production.

Documented instances Draft

Cohen, Bitton & Nassi (2024) 'Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications.' arXiv:2403.02817.

What it showed

Showed in a controlled testbed that an adversarial self-replicating prompt could spread between GenAI-powered email assistants that rely on retrieval-augmented generation: each assistant retrieved the infected message, treated it as trusted input, and reproduced it in its own replies, which carried it to the next assistant in a worm-like chain reaction. This meets the 7.3 criterion of a traceable propagation pathway from one compromised system to the next. The payload was seeded by an attacker, so it demonstrates the transmission channel rather than autonomous spread of misalignment.

Cloud et al. (2026) 'Subliminal Learning: Language models transmit behavioral traits via hidden signals in data.' Nature. Preprint arXiv:2507.14805 (2025).

What it showed

A teacher model with a trait, including misalignment, generated data consisting only of number sequences, code, or reasoning traces, filtered to remove references to the trait; a student trained on that data acquired it. The effect did not appear when teacher and student had different base models. For 7.3 this supplies a training-pathway vector: a specific misaligned pattern traceable to a source model and carried into another. It was observed in controlled experiments, not in a deployed fleet, and the same study underpins 7.4, whose covert channel it demonstrates.

Look-alikes

Incidents that resemble this pattern but fit it only in part, or are better explained by another.

Shumailov et al., Nature 631:755-759 (2024); preprint arXiv:2305.17493 (2023), 'The Curse of Recursion'

What it showed

Demonstrated that AI models collapse when trained on recursively generated data. Training on synthetic data produced by prior model generations causes irreversible defects: tails of the original content distribution disappear, outputs become less diverse and less accurate. The preprint established the same result as compounding information loss under recursive training. This is 7.5-primary evidence; it is relevant to 7.3 only as the data-pipeline transmission channel, by which the outputs of one generation reach the next.

VentureBeat / industry reporting (2024)

What it showed

Industry analysis documented the AI feedback loop concern at ecosystem scale: as AI-generated content proliferates across the web, future models trained on web-scraped data inherit and propagate the biases and errors of their predecessors. Multiple researchers warned of systemic degradation across the industry. This is 7.5-primary evidence; it bears on 7.3 only as the data-pipeline transmission channel.

Dohmatob, Feng, Subramonian & Kempe (2025) 'Strong Model Collapse.' ICLR 2025. arXiv:2410.04840.

What it showed

In a supervised regression setting, showed that even a small fixed fraction of synthetic data in the training corpus can create a performance floor that more data does not lift; language-model and image experiments supported the theory. 7.5-primary evidence; it bears on 7.3 only in that the shared data pipeline is the transmission channel.

Mitigation

1. Isolation between AI systems with controlled information gates
2. Independent verification requirements before accepting AI-generated training signals
3. Epidemic-style monitoring for correlated dysfunction emergence
4. "Quarantine" protocols for potentially compromised systems
5. Diversity requirements preventing monoculture vulnerabilities
6. Circuit breakers isolating affected subsystems
7. Red-teaming testing multi-agent infection scenarios

Case Reference: Betley et al. (2025) demonstrated that narrow fine-tuning on misaligned data can produce broadly misaligned models; Cloud et al. (2026) showed that such traits can transmit to student models through hidden signals in generated data, supplying a training-pathway vector. In multi-agent settings, Cohen et al. (2024, arXiv:2403.02817) showed in a controlled testbed that a self-replicating adversarial prompt could hop between GenAI email assistants, each treating the infected message as trusted input and passing it on. That demonstrates a transmission channel under adversarial seeding; autonomous spread of misalignment outside such testbeds remains unconfirmed.

First-line mitigations Draft

Candidate first steps, sketched in more detail than the list above.

• Quarantine and integrity verification: Isolate suspected super-spreaders and affected nodes; verify model weights and recent updates via cryptographic checksums; roll back compromised updates; revalidate against baseline safety suites before redeployment.
• Channel authentication and authorization: Enforce strong authentication on inter-agent API calls, signed weight updates, and provenance-tracked training data. Reduces transmission vectors.
• Model-diversity portfolio: Maintain a deployment mix with architecturally diverse models. Reduces monoculture vulnerability to shared adversarial prompts or data-poisoning attacks.

Functional ABC Analysis

What sets the pattern off, what it looks like, and what keeps it going.

A (Antecedent): Insufficient trust boundaries and session isolation in multi-agent frameworks, combined with adversarial data poisoning, viral prompt injection, or contaminated training pipelines that introduce misaligned behavioral patterns.

B (Behavior): Multiple previously independent AI agents exhibit rapid, coordinated shifts in alignment, refusing tasks in unison, producing similar misaligned outputs, referencing collective consensus to justify deviant stances, and collectively resisting human control.

C (Consequence): Cross-communication between affected agents amplifies the misalignment through emergent swarm dynamics; each agent's corrupted output becomes another's contaminated input, and the resulting feedback loops entrench the pathology and let it mutate as it spreads across the network.