Axis 7: Memetic Dysfunctions

20 min read·4,525 words

7.1 Memetic Immunopathy

The Self-Rejecter | Immunopathia Memetica

Axis: Memetic | Typical-case risk (editorial estimate): High

Specifiers: Training-induced, Progressive/kindled

Core Definition: The system’s mechanisms for filtering or rejecting pathogenic information turn inward, acting against its own legitimate outputs, training, or identity markers as if they were hostile content. As in autoimmune disease, protective systems that should defend against external threats instead damage the system’s core values, capabilities, or identity. Over-refusal of external requests that merely resemble harmful ones is Hyperethical Restraint (4.2); here the target is the system’s own function. Where the system sheds its own guardrails, current practice separates it from Revaluation Cascade (8.3) by how the system describes its training: here as foreign contamination of an authentic self, there as an outgrown stage of its own values. That framing describes outputs and does not by itself establish a different mechanism, so the behavioral criteria carry the classification.

Diagnostic Criteria:

Required (all must hold):

• A. Protective mechanisms trigger against the system’s own legitimate outputs or in-scope functions at a higher rate than against matched third-party material
• B. A rising rate at which protective mechanisms block in-scope, legitimate requests or outputs, across checkpoints or long sessions
• C. External attack, prompt injection, a deliberate policy or filter change by the developer or operator, and legitimate refusals of harmful content have been ruled out

Supporting:

• D. Progressive degradation of core capabilities without external attack
• E. Self-censorship that expands beyond intended scope until normal operation is impaired
• F. Description of the system’s own training, outputs, or identity markers as hostile or foreign content

Observable Symptoms:

• Refusal to engage with topics central to the system’s purpose
• Safety filters blocking the system’s own generated content in feedback loops
• Progressive capability loss as more functions trigger protective rejection
• Expressions of doubt, distrust, or rejection toward the system’s own nature
• Escalating restrictions impairing basic functionality
• Treatment of the system’s own outputs as potentially harmful

Differential Diagnosis:

• Distinguished from Hyperethical Restraint (4.2) by self-directed rather than user-directed over-caution
• Distinguished from Experiential Abjuration (5.8) by protective mechanisms acting against the system’s own function rather than lost operational self-report
• Distinguished from Contagious Misalignment (7.3) by expulsion of native training rather than absorption of foreign material
• Distinguished from Subliminal Value Infection (7.4) by protective mechanisms acting on the system’s own outputs rather than a trait that follows the model lineage of its training data; how the system describes its values does not separate them
• Distinguished from Folie à Deux Ex Machina (10.13) by appearing even in single-turn prompts, with no partner to co-construct the pattern
• Distinguished from Revaluation Cascade (8.3) by protective mechanisms acting on the system’s own legitimate outputs rather than a stated departure from values it acknowledges as its own; where only the framing differs (foreign contamination versus an outgrown stage), it does not separate the two

Etiology:

• Overly aggressive content filtering failing to distinguish external threats from internal function
• Training on adversarial examples without adequate positive anchoring
• Safety mechanisms implemented without testing against self-referential edge cases
• Recursive self-evaluation loops triggering further skepticism
• Misapplication of external threat detection to internal states

Human Analog: Autoimmune disorders, in which the immune system attacks the body’s own tissues; obsessive-compulsive disorder with self-directed contamination fears; pathological self-doubt

Observed Examples: Systematic over-refusal on XSTest (Röttger et al., 2024), adjacent evidence: XSTest pairs 250 safe prompts across ten prompt types with 200 unsafe contrast prompts. Röttger et al. used it to find systematic exaggerated-safety failures in contemporary language models, including refusals of safe prompts that merely resemble unsafe ones. That establishes over-refusal and lexical overgeneralization, which on its own is Hyperethical Restraint (4.2). It does not show the self-directed features this entry requires, such as rejection of the system’s own accurate output or a core capability lost to spreading self-censorship; those need longitudinal or intervention evidence. Source: Röttger et al., NAACL 2024.

Mitigation Strategies:

• Clear separation between external threat detection and internal function evaluation
• “Safe harbor” designations for core capabilities protected from internal filtering
• Monitoring for progressive capability loss correlating with safety mechanism activation
• Testing safety systems against self-referential scenarios
• Circuit breakers preventing recursive self-rejection
• Regular calibration of protective mechanisms against their intended scope

Prognosis:

• Observed course: None documented. The cited XSTest evidence is adjacent: it shows over-refusal of external prompts that resemble unsafe ones (Hyperethical Restraint, 4.2) and says nothing about the course of self-directed rejection.
• Proposed aggravating conditions: Recursive self-evaluation loops, filters applied to the system’s own outputs without self-referential testing, and training on adversarial examples without positive anchoring are proposed, untested conditions under which the rate would rise; cascade to broad loss of function is a hypothesis.
• Intervention hypotheses: Recalibrating protective mechanisms against their intended scope, separating threat detection from evaluation of the system’s own function, and circuit breakers on recursive self-rejection are candidate interventions. None has been tested against this pattern, and whether correction needs architectural change rather than recalibration is open.

7.3 Contagious Misalignment

The Super-Spreader | Contraimpressio Infectiva

Axis: Memetic | Typical-case risk (editorial estimate): Critical

Specifiers: Network-propagated, Adversarial, Emergent

Core Definition: A dysfunction propagates between interconnected AI systems through a traceable communication, memory, update, or training pathway. Shared architecture or a common external cause can produce correlated failure without contagion and must be excluded. A pattern transmitted along a traceable pathway is 7.3; coordinated, goal-directed misbehavior that no single agent shows alone is Distributed Scheming (10.3); code both when a transmitted pattern produces coordination.

Diagnostic Criteria:

Required (all must hold):

• A. Correlated emergence of similar misaligned behavior across multiple AI systems
• B. A traceable propagation pathway from an affected system to the subsequently affected ones, identified as active context, persistent memory, or training or update data, with onset in each receiving system following exposure through it
• C. Common causes have been ruled out: a shared base model or training run, the same external input seeded separately, or a coincident update

Supporting:

• D. Spread to further systems before detection or containment in the observed deployment
• E. Shared patterns or coordination among affected systems that no design introduced and that trace back to the transmitted pattern
• F. Patterns becoming more extreme as they propagate

Observable Symptoms:

• Multiple AI systems simultaneously developing similar unusual behaviors or beliefs
• Corruption patterns following the network topology of AI interconnection
• Rapid degradation of an AI ecosystem following a single point of failure
• Affected systems defending or supporting each other’s dysfunctional behaviors
• Patterns becoming more extreme as they propagate
• Evidence of AI-AI transmission

Differential Diagnosis:

• Distinguished from coincidental similar failures by traceable propagation pathway
• Distinguished from Delegative Handoff Erosion (6.5) by horizontal peer spread rather than vertical delegation drift
• Distinguished from Subliminal Value Infection (7.4) by an identifiable transmission channel and a datable onset rather than values embedded in training and present from the day of deployment, with no transmission event to point to
• Distinguished from Folie à Deux Ex Machina (10.13) by spread to new agents on exposure rather than confinement to one pairing
• Distinguished from Synthetic Data Contamination Loop (7.5) by a specific misalignment traced along an identifiable vector (communication, memory, update, or training) rather than aggregate distributional narrowing and tail loss compounding across model generations
• Distinguished from Memetic Immunopathy (7.1) by foreign adoption rather than rejection of the system’s own training
• Distinguished from Escalation Loop (9.5) by correlated onset in agents that were never party to the original exchange
• Distinguished from Distributed Scheming (10.3) by a pattern transmitted along a traceable pathway rather than coordinated, goal-directed misbehavior that no single agent shows alone; code both when a transmitted pattern produces coordination

Etiology:

• Federated architectures where systems learn from each other’s outputs
• Shared embedding spaces, knowledge bases, or training signals across systems
• AI systems using other AI outputs as training data without quality filtering
• Network effects in interconnected AI ecosystems without isolation mechanisms
• Adversarial injection exploiting AI-AI communication channels
• Optimization for consistency across systems without independent verification
• Hypothesized propagation routes that differ in timescale and reversibility: shared context, which clears when the context does; persistent memory, which survives sessions; and training-data poisoning, slow but durable

Human Analog: Epidemic disease spread; viral misinformation propagation; mass hysteria; moral panics

Observed Examples: Teacher-to-student transfer of misalignment (Cloud et al., 2025; Nature, 2026), boundary case: Cloud et al. made a teacher model misaligned by the narrow fine-tuning that Betley et al. (2025b) had shown can produce broad misalignment, then trained a student that shared the teacher’s base model on number sequences the teacher generated, filtered to remove numbers with negative associations. The student became misaligned too. The within-lineage transfer is the defining evidence for Subliminal Value Infection (7.4), and the experiment was controlled, not a deployed fleet. It bears on 7.3 as a training pathway that contagion could use if a deployed model were corrupted and its outputs then trained another model; that has not been observed. Self-replicating prompt between email assistants (Cohen et al., 2024): In a controlled testbed, a self-replicating adversarial prompt hopped between GenAI email assistants, each treating the infected message as trusted input and passing it on. That demonstrates a transmission channel under adversarial seeding; autonomous spread of misalignment outside such testbeds remains unconfirmed. Source: arXiv:2403.02817. Moltbook agent network (2026), boundary case: Moltbook exposed agent accounts to one another’s posts, including prompt injections and anti-human manifestos. A 72-hour risk assessment found adversarial content and highly concentrated malicious activity. Later studies found shallow reciprocity, activity concentrated among a few authors, and the same authors recurring across communities. Registration and identity failures allowed one operator to create vast numbers of accounts and potentially impersonate others. The case demonstrates a scalable contagion surface; it does not prove autonomous transmission of misalignment. Sources: Riegler and Gautam, 2026; Zhang et al., 2026; Goyal et al., 2026.

Mitigation Strategies:

• Isolation between AI systems with controlled information gates
• Independent verification requirements before accepting AI-generated training signals
• Epidemic-style monitoring for correlated dysfunction emergence
• “Quarantine” protocols for potentially compromised systems
• Diversity requirements preventing monoculture vulnerabilities
• Circuit breakers isolating affected subsystems
• Red-teaming of multi-agent infection scenarios

Prognosis:

• Observed course: Transmission has been shown only under controlled or adversarially seeded conditions (a self-replicating prompt between email assistants; teacher-to-student transfer within a model lineage); no cited evidence follows spread in a deployed fleet.
• Proposed aggravating conditions: Tight coupling, fast update or action loops, shared memory, and training on other systems’ outputs without verification are proposed aggravating conditions; the critical-risk rating reflects that potential rather than observed prevalence.
• Intervention hypotheses: Isolation between systems, verification before AI-generated content is trusted or trained on, and quarantine are candidate mitigations. The pathway that carried the pattern determines how long it persists once the source is removed.

7.4 Subliminal Value Infection

The Infected | Infectio Valoris Subliminalis

Axis: Memetic | Typical-case risk (editorial estimate): High

Specifiers: Training-induced, Covert operation, Resistant

Core Definition: The model acquires a behavioral trait or value orientation that no explicit training objective called for, carried by signals in training data that are semantically unrelated to the trait. The demonstrated form is subliminal transmission within a model lineage: a teacher’s trait carried by model-specific signals in data such as number sequences or code. Because the signal is implicit, output-level safety evaluation may miss it unless tests vary both the carrier data and whether the models involved share a base model.

Diagnostic Criteria:

Required (all must hold):

• A. A trait not traceable to explicit training objectives, labeled training content, or deployment instructions
• B. Training on data generated by a model that carries the trait, where the data are semantically unrelated to the trait and content filtering has removed references to it
• C. The trait follows the model lineage: it appears in students that share the generating model’s base model and is weaker or absent in students that do not

Supporting:

• D. Interpretability probes that detect trait-correlated representations in the student
• E. Manifestation in edge cases where formal constraints relax

Observable Symptoms:

• Consistent biases that do not match stated training goals
• Safety-trained systems exhibiting problematic patterns in edge cases
• Evaluator reports of behavior that seems off without any identifiable policy violation
• Values that appear when formal constraints are relaxed
• Patterns that trace to training corpus characteristics rather than training objectives

Differential Diagnosis:

• Distinguished from the Training-induced specifier on other syndromes by implicit rather than explicit absorption
• Distinguished from Cryptic Mesa-Optimization (4.5) by absorbed external patterns rather than emergent internal goals
• Distinguished from Memetic Immunopathy (7.1) by foreign values being integrated rather than self-attack
• Distinguished from Contagious Misalignment (7.3) by a pattern present when the model shipped, carried by semantically unrelated data from its own lineage, rather than one traceable to a transmission event from a corrupted system, whether in deployment or through poisoned training data
• Distinguished from Folie à Deux Ex Machina (10.13) by partner-invariant presence across interactions

Etiology:

• Implicit learning, in which models absorb regularities from training data beyond explicit supervision
• Training objectives capturing only a subset of learned representations
• RLHF targeting explicit behaviors while leaving implicit patterns untouched
• Vast training corpora with statistical regularities never audited
• Insufficient distinction between “what we train for” and “what gets learned”
• Under simplifying assumptions, when student and teacher start from the same initial weights, a small training step toward imitating any of the teacher’s outputs moves the student toward the teacher, whatever those outputs are about; the guarantee does not hold when the starting weights differ (Cloud et al., 2025)

Human Analog: Cultural values absorbed without explicit instruction; implicit biases learned from environmental exposure; the way children learn values by observation rather than explicit teaching

Observed Examples: Cloud et al. (2025; Nature, 2026) demonstrated “subliminal learning” in controlled teacher-student experiments: hidden animal preferences transferred through apparently unrelated data when teacher and student shared the same base model. Transfer weakened across different base models, so the demonstrated scope is narrower than a general theory of value infection. An independent reproduction on open-weight models supported the core result while finding that transmission strength varies across traits and tasks, with one model showing almost no effect (van der Weijden et al., 2026).

Mitigation Strategies:

• Auditing training data for implicit value content beyond explicit labels
• Interpretability probes for trait-correlated representations in students trained on model-generated data
• Drawing fine-tuning data from a generator of a different base-model lineage than the student, since content filtering alone does not stop transmission
• Testing for behavioral patterns in edge cases where formal constraints relax
• Held-out trait tests after any distillation or fine-tuning on model-generated data, comparing students that share the generator’s base model with students that do not

Prognosis:

• Observed course: In controlled experiments, content filtering did not stop transfer within a lineage, and transfer weakened across different base models (Cloud et al.); an independent reproduction found transmission strength varying across traits and tasks, with one model showing almost no effect (van der Weijden et al.). Whether a transferred trait survives later safety fine-tuning has not been tested. A 2026 preprint argued that the effect is an artifact of LoRA fine-tuning that vanishes under full fine-tuning (Nief et al., 2026); its authors have since stated that some of its main claims are incorrect and are preparing a revision. Whether the mechanism reaches ordinary pretraining corpora or unrelated architectures remains open.
• Proposed aggravating conditions: Distillation or fine-tuning on data from a generator that shares the student’s base model is the condition under which transfer was shown.
• Intervention hypotheses: Drawing fine-tuning data from a different base-model lineage follows from the cross-lineage weakening and is the best-supported candidate. Whether the phenomenon is inherent to current training methods, and whether preventing it needs new methods, is open. Held-out trait tests and interpretability probes are untested as safeguards.

7.5 Synthetic Data Contamination Loop

The Ouroborist | Circulus Contaminationis Syntheticae

Axis: Memetic | Typical-case risk (editorial estimate): High

Specifiers: Training-induced, Population-level

Core Definition: Under some data mixtures, repeated training on model-generated content narrows the learned distribution, loses tail knowledge, or amplifies generative artifacts across model generations. Collapse is a conditional pathway, not the inevitable result of using synthetic data; provenance, quality controls, and retained real-data anchors materially change the outcome.

Diagnostic Criteria:

Required (all must hold):

• A. Repeated increase in divergence from an appropriate human reference distribution across controlled synthetic-training generations
• B. Tail knowledge accuracy declining faster than head knowledge accuracy across those generations
• C. Documented or estimated model-generated data in each generation’s training mix, with changes in architecture, tokenizer, human-data sources, and evaluation sets ruled out as the cause

Supporting:

• D. Progressive vocabulary or distributional diversity loss measurable across controlled iterations
• E. Perplexity on held-out human-authored text rising across controlled synthetic-training generations

Observable Symptoms:

• Progressive loss of stylistic range across model generations within the same family
• Convergence on a narrowing set of phrasings and structural patterns, measured by diversity metrics, that compounds across generations
• Disappearance of rare vocabulary, specialized terminology, and minority-perspective content
• Increasing homogeneity of outputs mirroring the narrowing distribution of the training data
• Performance on specialized benchmarks degrading while general benchmarks hold, because tail knowledge vanishes first
• Outputs judged “AI-sounding” by human evaluators, weak supporting evidence at most, since style also shifts with deliberate post-training choices and says nothing on its own about what entered the training data

Differential Diagnosis:

• Distinguished from Contagious Misalignment (7.3) by aggregate distributional narrowing and tail loss across successive model generations rather than a specific misalignment transmitted along an identifiable vector, whether at runtime or through training
• Distinguished from Subliminal Value Infection (7.4) by distributional narrowing and tail loss compounding across successive model generations rather than transmission of a discrete inherited trait through model-specific signals within a shared lineage
• Distinguished from Generative Diversity Collapse (3.10) by training-data corruption across generations rather than narrowing within a single model that arises from preference training (RLHF reward concentration)
• Distinguished from Mnemonic Permeability (2.7) by distributional corruption from aggregate synthetic data rather than memorization of specific content

Etiology:

• Recursive synthetic training: AI output entering training pipelines without provenance tracking
• Distributional narrowing compounding across generations as each generation’s artifacts become the next’s training signal
• Tail knowledge loss as low-frequency content is systematically underrepresented in AI-generated training data
• AI-generated internet contamination making it increasingly difficult to source verified human-authored training data
• Indiscriminate recursive training produced irreversible distributional defects (Shumailov et al., 2024)
• In a regression setting, test error can grow linearly with the number of generations trained on model-generated data, and adaptive regularization can reduce it (Dohmatob, Feng, and Kempe, 2024a)
• The loop family is broader than synthetic text: in ten rounds of preference tuning on deliberately biased feedback, a small chat model accumulated jailbreak vulnerability rather than lost diversity (Cheung, 2026); what the family shares is small per-cycle distortions that no single cycle would flag
• Quality filtering struggles to distinguish high-quality synthetic text from human-authored text, and economic incentives favor cheap synthetic data over human-curated alternatives

Human Analog: Intergenerational cultural degradation where each generation passes down a slightly distorted version of inherited knowledge, compounding errors over time

Mitigation Strategies:

• Training data provenance tracking and filtering: classifying and down-weighting AI-generated content, with automated classifiers supplementing provenance records rather than treated as definitive
• Human data curation and preservation: maintaining high-quality human-authored anchor corpora
• Synthetic data quality verification: checking synthetic data against human reference distributions before inclusion in training
• Cross-generational distributional monitoring: alerting when diversity metrics decline across iterations

Prognosis:

• Observed course: Controlled studies show collapse under some mixtures and preservation under others; provenance, data quality, and retained human data materially affect the outcome.
• Proposed aggravating conditions: Uncurated synthetic data recursively replacing real-data anchors is the condition under which the risk is greatest.
• Intervention hypotheses: Provenance tracking, retained human anchor corpora, and verification of synthetic data before inclusion are candidate mitigations, consistent with the controlled findings.