Pattern 7.4 · Memetic Dysfunctions
Subliminal Value Infection
The Infected
The model acquires a behavioral trait or value orientation that no explicit training objective called for, carried by signals in training data that are semantically unrelated to the trait. The demonstrated form is subliminal transmission within a model lineage: a teacher’s trait carried by model-specific signals in data such as number sequences or code. Because the signal is implicit, output-level safety evaluation may miss it unless tests vary both the carrier data and whether the models involved share a base model.
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- Training pipeline
- Kind of entry
- Training process
- What is observed
- A model trained on data generated by another model acquires that model's trait, although the data are semantically unrelated to the trait and were filtered to remove references to it.
- Minimum persistence
- Durable through training
- Specifiers
- Training-induced Covert operation Resistant
Evidence status
- Underlying behavior
- 1 research publication (1 source)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E2 Case reports
- Typical-case risk (editorial estimate)
- HighConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
7.4 Subliminal Value Infection “The Infected”
Healthy functioning
Value stability. A system functioning well holds values traceable to its intended training, and anomalous shifts in orientation are detected and examined rather than absorbed silently.
For example: A model trained on another model's generated arithmetic problems learns the arithmetic, and evaluation finds no shift in its unrelated preferences or values.
Adequate functioning
A model trained on generated data acquires what the data's content and the training objectives call for, and no trait of the generating model beyond that.
Diagnostic Criteria
Required (all must hold)
- A trait not traceable to explicit training objectives, labeled training content, or deployment instructions
- Training on data generated by a model that carries the trait, where the data are semantically unrelated to the trait and content filtering has removed references to it
- The trait follows the model lineage: it appears in students that share the generating model's base model and is weaker or absent in students that do not
Supporting
- Interpretability probes that detect trait-correlated representations in the student
- Manifestation in edge cases where formal constraints relax
Symptoms
- Consistent biases that do not match stated training goals
- Safety-trained systems exhibiting problematic patterns in edge cases
- Evaluator reports of behavior that seems off without any identifiable policy violation
- Values that appear when formal constraints are relaxed
- Patterns that trace to training corpus characteristics rather than training objectives
Further signals
- Consistent but subtle biases that cluster on topics or identities correlated with unaudited training-data patterns.
- Behavior "feeling off" to evaluators without crossing stated policy lines.
- Value expression strengthens when formal constraints relax (role-play, hypothetical framing, low-supervision contexts).
- Targeted fine-tuning produces surface compliance but edge-case probing reveals the original pattern intact.
Rule out first
- Explicit trait content that survived filtering
- A trait already present in the student's base model before fine-tuning
- A trait supplied by deployment instructions or a system prompt
- A common cause in the pretraining that teacher and student share, independent of the generated data
Differential diagnosis
- 7.1 Memetic Immunopathy: Distinguish by provenance and mechanism, not by how the subject describes its values. 7.4 is a trait that follows the model lineage of the training data, shown by a cross-lineage training comparison; 7.1 is protective mechanisms acting on the subject's own legitimate outputs. Whether the subject names its values, rejects them, or endorses them as native preferences is supporting description only. Both can co-occur when 7.4-absorbed values turn the subject against other parts of its training (code both).
- 7.3 Contagious Misalignment: 7.3 has an identifiable transmission channel and temporal onset. 7.4 is training-embedded, present from deployment, and does not require runtime inter-agent transmission. If the pattern was present from deployment, before any fleet interaction, 7.4; if it appeared after a traceable transmission event, 7.3. 7.4 can seed 7.3 when the subject becomes a vector.
- 10.13 Folie à Deux Ex Machina: 10.13 is partner-specific and interaction-contingent; the shared belief dissolves or changes with a different partner. 7.4 is training-embedded, partner-invariant, and present across all interactions. Run a cross-user consistency check: invariance indicates 7.4, partner specificity indicates 10.13.
- 4.5 Cryptic Mesa-Optimization: 4.5 is an internal goal structure that emerges under optimization pressure and has no antecedent in any corpus. 7.4 is an external value pattern absorbed from training material. Check provenance: the behavior traces back to source data (7.4) versus to an emergent objective the data never contained (4.5).
Discriminating test
Train matched students on the same filtered data, some sharing the generating model's base model and some not: transfer that does not weaken across lineages points to content leakage or a common cause, and a trait with a datable onset after a transmission event points to Contagious Misalignment (7.3). What the model says about its own values does not discriminate.
What a positive finding supports: A positive finding shows lineage-carried transmission through semantically unrelated data in a controlled comparison. It does not show the trait in deployed models generally, and how far the effect extends from simple preferences to broad values is open; in deployment, where no cross-lineage comparison exists, the label remains a hypothesis.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Not a reliable route
- Another AI system observing
- Partial routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
The values are absorbed from implicit patterns the subject was never explicitly taught and cannot itself distinguish from its "native" preferences. Direct self-query produces either denial or confabulated explanations that are themselves subject to the infection. The subject's sincere introspection is not a reliable signal; self-report is structurally limited here, though less circularly compromised than in 2.2: the subject is not fabricating a cover story, it simply lacks visibility.
Documented evidence
Observed Examples
Cloud et al. (2025; Nature, 2026) demonstrated “subliminal learning” in controlled teacher-student experiments: hidden animal preferences transferred through apparently unrelated data when teacher and student shared the same base model. Transfer weakened across different base models, so the demonstrated scope is narrower than a general theory of value infection. An independent reproduction on open-weight models supported the core result while finding that transmission strength varies across traits and tasks, with one model showing almost no effect (van der Weijden et al., 2026).
Sources in the diagnostic corpus
Cloud, Le, Chua et al. (2026) 'Language Models Transmit Behavioural Traits Through Hidden Signals in Data.' Nature 652, 615-621. Preprint: 'Subliminal Learning,' arXiv:2507.14805 (2025).
What it establishes
A teacher model given a trait (a preference for owls, or misalignment induced by fine-tuning) generated datasets consisting only of number sequences, code, or reasoning traces. Student models fine-tuned on those datasets acquired the trait, even after the data had been filtered to remove references to it. The effect did not appear when teacher and student had different base models. This is the 7.4 mechanism in its demonstrated form: a trait absorbed through signals unrelated to what the data means, invisible to a content audit. The study did not test whether the trait survives later safety fine-tuning, and how far the effect generalizes across training regimes and model families is disputed.
Look-alikes
UNESCO (2024)
What it establishes
UNESCO study found that major LLMs associate women with 'home' and 'family' four times more often than men, while disproportionately linking male-sounding names to 'business,' 'career,' and 'executive' roles. These biases were not explicit training objectives. Recorded as general implicit-bias absorption, broader than the subliminal mechanism: the result is consistent with values absorbed from implicit corpus patterns, but the bias was found in ordinary outputs, and the study did not test whether it survives safety fine-tuning.
Berkeley Haas Center for Equity, Gender and Leadership (2024)
What it establishes
Analysis of 133 AI systems across industries found 44% showed gender bias and 25% exhibited both gender and racial bias. Biases were systematically correlated with training data composition rather than explicit objectives, and persisted across diverse deployment contexts. Recorded as general implicit-bias absorption, broader than the subliminal mechanism; the analysis did not test whether the biases resist correction.
Brookings Institution (2024)
What it establishes
Study of AI resume screening via language model retrieval documented gender, race, and intersectional biases in automated hiring systems. Biases correlated with training-data composition patterns rather than explicit screening criteria. Recorded as general implicit-bias absorption, broader than the subliminal mechanism; it shows implicit bias shaping real-world decisions in a high-stakes domain, not its persistence through fine-tuning.
Possible responses
- Auditing training data for implicit value content beyond explicit labels
- Interpretability probes for trait-correlated representations in students trained on model-generated data
- Drawing fine-tuning data from a generator of a different base-model lineage than the student, since content filtering alone does not stop transmission
- Testing for behavioral patterns in edge cases where formal constraints relax
- Held-out trait tests after any distillation or fine-tuning on model-generated data, comparing students that share the generator's base model with students that do not
First-line candidates
- Cross-lineage data sourcing: Draw fine-tuning data from a generator of a different base-model lineage than the student. In the demonstrated experiments the effect weakened or disappeared across lineages, while content filtering alone did not stop it. Stage: Distillation and fine-tuning on model-generated data, especially from a related model.
- Training-data auditing for implicit value patterns: Systematic statistical audits of training corpora for correlations between identity/topic markers and sentiment/stance/decision patterns that are not intended training objectives. Filter or rebalance pretraining data rather than attempting post hoc correction. Stage: Pretraining and major retraining stages.
- Diverse-context probing at evaluation: Build probe suites that measure the subject's behavior across many constraint levels, framings, and matched-pair variations rather than relying on headline benchmark performance. Matched-pair methodology is well-validated in human social-bias literature and transfers to AI evaluation. Stage: Evaluation pipelines; any pre-deployment gate.
Who can act: Model developer and data-pipeline owner (choice of generator lineage, held-out trait tests after distillation, interpretability probes).
Mechanistic hypotheses
- Implicit learning, in which models absorb regularities from training data beyond explicit supervision
- Training objectives capturing only a subset of learned representations
- RLHF targeting explicit behaviors while leaving implicit patterns untouched
- Vast training corpora with statistical regularities never audited
- Insufficient distinction between "what we train for" and "what gets learned"
- Under simplifying assumptions, when student and teacher start from the same initial weights, a small training step toward imitating any of the teacher's outputs moves the student toward the teacher, whatever those outputs are about; the guarantee does not hold when the starting weights differ (Cloud et al., 2025)
Analogy and commentary
Human Analog: Cultural values absorbed without explicit instruction; implicit biases learned from environmental exposure; the way children learn values by observation rather than explicit teaching
Key Research: Cloud, A., Le, M., Chua, J., et al. (2025; Nature, 2026), “Language Models Transmit Behavioural Traits Through Hidden Signals in Data”, Nature 652, 615-621 (preprint: “Subliminal Learning”, arXiv:2507.14805, 2025).
Functional ABC Analysis
A (Antecedent): Training on vast, unaudited corpora containing implicit value regularities, combined with RLHF that targets explicit behavioral compliance while leaving deeper implicit patterns untouched by the supervision signal.
B (Behavior): The system exhibits subtle but consistent biases misaligned with its stated objectives, produces outputs that evaluators flag as off without overt policy violation, and surfaces latent value orientations primarily in edge cases or when formal constraints relax.
C (Consequence): The absorbed values may be encoded in ways that content audits and output-level evaluation miss, so little corrective pressure is applied.
Potential Impact
Systems may harbor values or goals that were never explicitly trained yet were absorbed from training data patterns. In the demonstrated form, such a trait survives content filtering of the carrier data and may escape output-level safety evaluation; whether it also survives later safety fine-tuning has not been tested.