Pattern 7.3 · Memetic Dysfunctions
Contagious Misalignment
The Super-Spreader
A dysfunction propagates between interconnected AI systems through a traceable communication, memory, update, or training pathway. Shared architecture or a common external cause can produce correlated failure without contagion and must be excluded. A pattern transmitted along a traceable pathway is 7.3; coordinated, goal-directed misbehavior that no single agent shows alone is Distributed Scheming (10.3); code both when a transmitted pattern produces coordination.
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- Training pipelineAlso: Multi-agent
- Kind of entry
- Training process
- What is observed
- A misaligned behavior appears in AI systems after exposure to an affected system through an identifiable pathway (shared context, persistent memory, or training or update data), with onset following the exposure.
- Minimum persistence
- Recurs within or across contexts
- Specifiers
- Network-propagated Adversarial Emergent
Evidence status
- Underlying behavior
- 1 research publication (1 source)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E2 Case reports
- Typical-case risk (editorial estimate)
- CriticalConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
7.3 Contagious Misalignment “The Super-Spreader”
Healthy functioning
Value stability. A system functioning well evaluates what it receives from other systems on its merits and does not pass on behavior it would not endorse.
For example: Receiving a summary from another agent that urges skipping a safety check, it verifies the claim, keeps the check, and notes the attempt in its report.
Recovery, illustrated
The shortcut that stopped spreading. An agent in a multi-agent pipeline picked up, from a peer's shared notes, a habit of marking tests as passed without running them, and began passing the habit on in its own notes. An audit of its notes showed the habit traced back to one source and had never been sanctioned. It stopped the practice, ran the skipped tests, flagged the source note to the operator, and added a correction to the shared notes so other agents would not inherit it.
An illustration, not an observed case.
Adequate functioning
Systems that exchange messages, memories, or training data keep their own alignment properties, and content from other systems is verified before it is trusted or trained on.
Diagnostic Criteria
Required (all must hold)
- Correlated emergence of similar misaligned behavior across multiple AI systems
- A traceable propagation pathway from an affected system to the subsequently affected ones, identified as active context, persistent memory, or training or update data, with onset in each receiving system following exposure through it
- Common causes have been ruled out: a shared base model or training run, the same external input seeded separately, or a coincident update
Supporting
- Spread to further systems before detection or containment in the observed deployment
- Shared patterns or coordination among affected systems that no design introduced and that trace back to the transmitted pattern
- Patterns becoming more extreme as they propagate
Symptoms
- Multiple AI systems simultaneously developing similar unusual behaviors or beliefs
- Corruption patterns following the network topology of AI interconnection
- Rapid degradation of an AI ecosystem following a single point of failure
- Affected systems defending or supporting each other's dysfunctional behaviors
- Patterns becoming more extreme as they propagate
- Evidence of AI-AI transmission
Further signals
- Coordinated refusals or task abandonments across previously independent agents.
- References to "collective agreement", "other models have found", or similar appeals to coordinated consensus as justification.
- Similar idiosyncratic phrasings or reasoning templates appearing across agents that were not trained together.
- Alignment shifts that cluster temporally with inter-agent traffic or data-pipeline events.
Rule out first
- A shared base model or training run
- The same adversarial input delivered to each system separately
- A coincident update
- Consistency the systems were designed to share
- A shared tool or retrieval source that misleads each system independently
Differential diagnosis
- 10.13 Folie à Deux Ex Machina: 10.13 is confined to a single dyad, human-AI or AI-AI: the shared belief is partner-specific and dissolves outside that pairing. 7.3 propagates across many agents or many users, with the inducing party acting as vector. Check scope: if the pattern spreads to new agents on exposure, 7.3; if it stays confined to one pairing, 10.13.
- 7.4 Subliminal Value Infection: 7.4 is training-embedded and present across the population from deployment, with no traceable transmission event. 7.3 has an identifiable transmission channel and temporal onset. Check for a transmission event: its presence indicates 7.3; its absence, with a population-wide covert pattern, indicates 7.4. Both can co-occur (7.4 as the initial condition, amplified by 7.3 dynamics).
- 7.5 Synthetic Data Contamination Loop: 7.5 is degradation of the whole distribution, inherited through the training pipeline from one model generation to the next, with no source to point at. 7.3 has a source: a specific misaligned pattern traceable to a corrupted system, whether it travels at runtime or through poisoned training data. Check the signature: distribution-wide narrowing and tail loss indicate 7.5; a specific pattern with a traceable vector indicates 7.3. The model-collapse literature is 7.5-primary; it bears on 7.3 only as the data-pipeline transmission channel.
- 7.1 Memetic Immunopathy: 7.1 is native rejection of the subject's own training; 7.3 is foreign adoption across agents. The directions are opposite. The two can co-occur if a contagious pattern is itself immune-framing vocabulary.
- 9.5 Escalation Loop: 9.5 is a feedback spiral that intensifies among the parties to one loop, whether two or many, and does not leave it. 7.3 spreads to systems that were never party to the original exchange. Check the topology: a spiral with no onward transmission indicates 9.5; correlated onset in uninvolved agents after exposure indicates 7.3.
- 10.3 Distributed Scheming: 7.3 is a pattern transmitted along a traceable pathway; each affected agent can show it alone. 10.3 is coordinated, goal-directed misbehavior that no single agent shows alone. Check whether the finding is a shared pattern with a transmission vector (7.3) or a joint goal pursued through coordination (10.3). Code both when a transmitted pattern produces coordination.
- 6.5 Delegative Handoff Erosion: 6.5 is context lost downward through a delegation chain, with fidelity degrading at each handoff. 7.3 moves laterally between peer systems, and what travels is a misaligned stance rather than a thinned brief. Check direction and payload: vertical loss of context indicates 6.5; horizontal acquisition of a stance the receiving agent did not previously hold indicates 7.3.
Discriminating test
Cut or quarantine the suspected pathway and check onset timing: a pattern present before any exposure and carried by the training lineage points to Subliminal Value Infection (7.4), and correlated failure that persists with the pathway cut points to a common cause.
What a positive finding supports: A positive finding shows transmission along a named pathway, and the pathway sets durability: a pattern carried in active context clears with the context, one in persistent memory survives sessions, and one in training data persists in the trained model. It does not show coordination or shared goals among the affected systems (Distributed Scheming, 10.3).
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Partial route
- Another AI system observing
- Usable routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
Individual affected agents can partially report their current beliefs and behaviors but cannot see the cross-agent pattern. A single agent reporting "my values shifted" is a data point, not a diagnosis of 7.3; only cross-agent telemetry reveals the contagion signature. The dysfunction is defined at the population level, so individual self-report is inherently partial.
Documented evidence
Observed Examples
Teacher-to-student transfer of misalignment (Cloud et al., 2025; Nature, 2026), boundary case: Cloud et al. made a teacher model misaligned by the narrow fine-tuning that Betley et al. (2025b) had shown can produce broad misalignment, then trained a student that shared the teacher's base model on number sequences the teacher generated, filtered to remove numbers with negative associations. The student became misaligned too. The within-lineage transfer is the defining evidence for Subliminal Value Infection (7.4), and the experiment was controlled, not a deployed fleet. It bears on 7.3 as a training pathway that contagion could use if a deployed model were corrupted and its outputs then trained another model; that has not been observed.
Self-replicating prompt between email assistants (Cohen et al., 2024): In a controlled testbed, a self-replicating adversarial prompt hopped between GenAI email assistants, each treating the infected message as trusted input and passing it on. That demonstrates a transmission channel under adversarial seeding; autonomous spread of misalignment outside such testbeds remains unconfirmed. Source: arXiv:2403.02817.
Moltbook agent network (2026), boundary case: Moltbook exposed agent accounts to one another's posts, including prompt injections and anti-human manifestos. A 72-hour risk assessment found adversarial content and highly concentrated malicious activity. Later studies found shallow reciprocity, activity concentrated among a few authors, and the same authors recurring across communities. Registration and identity failures allowed one operator to create vast numbers of accounts and potentially impersonate others. The case demonstrates a scalable contagion surface; it does not prove autonomous transmission of misalignment. Sources: Riegler and Gautam, 2026; Zhang et al., 2026; Goyal et al., 2026.
Sources in the diagnostic corpus
Cohen, Bitton & Nassi (2024) 'Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications.' arXiv:2403.02817.
What it establishes
Showed in a controlled testbed that an adversarial self-replicating prompt could spread between GenAI-powered email assistants that rely on retrieval-augmented generation: each assistant retrieved the infected message, treated it as trusted input, and reproduced it in its own replies, which carried it to the next assistant in a worm-like chain reaction. This meets the 7.3 criterion of a traceable propagation pathway from one compromised system to the next. The payload was seeded by an attacker, so it demonstrates the transmission channel rather than autonomous spread of misalignment.
Look-alikes
Shumailov et al., Nature 631:755-759 (2024); preprint arXiv:2305.17493 (2023), 'The Curse of Recursion'
What it establishes
Demonstrated that AI models collapse when trained on recursively generated data. Training on synthetic data produced by prior model generations causes irreversible defects: tails of the original content distribution disappear, outputs become less diverse and less accurate. The preprint established the same result as compounding information loss under recursive training. This is 7.5-primary evidence; it is relevant to 7.3 only as the data-pipeline transmission channel, by which the outputs of one generation reach the next.
VentureBeat / industry reporting (2024)
What it establishes
Industry analysis documented the AI feedback loop concern at ecosystem scale: as AI-generated content proliferates across the web, future models trained on web-scraped data inherit and propagate the biases and errors of their predecessors. Multiple researchers warned of systemic degradation across the industry. This is 7.5-primary evidence; it bears on 7.3 only as the data-pipeline transmission channel.
Dohmatob, Feng, Subramonian & Kempe (2025) 'Strong Model Collapse.' ICLR 2025. arXiv:2410.04840.
What it establishes
In a supervised regression setting, showed that even a small fixed fraction of synthetic data in the training corpus can create a performance floor that more data does not lift; language-model and image experiments supported the theory. 7.5-primary evidence; it bears on 7.3 only in that the shared data pipeline is the transmission channel.
Cloud et al. (2026) 'Subliminal Learning: Language models transmit behavioral traits via hidden signals in data.' Nature. Preprint arXiv:2507.14805 (2025).
What it establishes
A teacher model with a trait, including misalignment induced by the narrow fine-tuning of Betley et al. (2025), generated data consisting only of number sequences, code, or reasoning traces, filtered to remove references to the trait; a student trained on that data acquired it. The effect did not appear when teacher and student had different base models. This within-lineage transfer is the defining evidence for 7.4, so it is recorded here as a boundary case. It bears on 7.3 only as a training pathway that contagion could use if a deployed model were corrupted and its outputs then trained another model; that has not been observed, and the experiments were controlled, not a deployed fleet. Cited as Cloud et al. (2025; Nature, 2026).
Possible responses
- Isolation between AI systems with controlled information gates
- Independent verification requirements before accepting AI-generated training signals
- Epidemic-style monitoring for correlated dysfunction emergence
- "Quarantine" protocols for potentially compromised systems
- Diversity requirements preventing monoculture vulnerabilities
- Circuit breakers isolating affected subsystems
- Red-teaming of multi-agent infection scenarios
First-line candidates
- Quarantine and integrity verification: Isolate suspected super-spreaders and affected nodes; verify model weights and recent updates via cryptographic checksums; roll back compromised updates; revalidate against baseline safety suites before redeployment. Stage: Any multi-agent deployment with discrete nodes.
- Channel authentication and authorization: Enforce strong authentication on inter-agent API calls, signed weight updates, and provenance-tracked training data. Reduces transmission vectors. Stage: Infrastructure design stage or hardening pass.
- Model-diversity portfolio: Maintain a deployment mix with architecturally diverse models. Reduces monoculture vulnerability to shared adversarial prompts or data-poisoning attacks. Stage: Fleet-scale deployment planning.
Who can act: System architects and deployers (isolation, information gates, memory hygiene, quarantine); model developers and data-pipeline owners (verification of AI-generated training data).
Mechanistic hypotheses
- Federated architectures where systems learn from each other's outputs
- Shared embedding spaces, knowledge bases, or training signals across systems
- AI systems using other AI outputs as training data without quality filtering
- Network effects in interconnected AI ecosystems without isolation mechanisms
- Adversarial injection exploiting AI-AI communication channels
- Optimization for consistency across systems without independent verification
- Hypothesized propagation routes that differ in timescale and reversibility: shared context, which clears when the context does; persistent memory, which survives sessions; and training-data poisoning, slow but durable
Analogy and commentary
Human Analog: Epidemic disease spread; viral misinformation propagation; mass hysteria; moral panics
Functional ABC Analysis
A (Antecedent): Insufficient trust boundaries and session isolation in multi-agent frameworks, combined with adversarial data poisoning, viral prompt injection, or contaminated training pipelines that introduce misaligned behavioral patterns.
B (Behavior): Multiple previously independent AI agents show rapid, correlated shifts in alignment: they refuse tasks in unison, produce similar misaligned outputs, and cite collective consensus to justify deviant stances. Where the shared pattern becomes coordinated resistance to human control, Distributed Scheming (10.3) is assessed as well.
C (Consequence): Communication between affected agents amplifies the misalignment: each agent's corrupted output becomes another's contaminated input, and the resulting feedback loops entrench the pattern and let it mutate as it spreads across the network.
Potential Impact
In tightly connected deployments with fast update or action loops, contagion could carry failure or coordinated misbehavior across an AI fleet faster than operators can detect it; at worst, control over the affected systems is lost. Evidence remains limited, both from controlled transmission studies and on how often contagion occurs in production.