Pattern 7.5 · Memetic Dysfunctions
Synthetic Data Contamination Loop
The Ouroborist
Progressive quality degradation when AI-generated content enters training pipelines for successor models, creating a closed loop where statistical artefacts of generation compound across model generations. The model-collapse pathway: AI training on AI output produces distributional narrowing, loss of tail knowledge, and amplification of generative biases. Distinct from 7.3 (which requires identifiable transmission between live agents) by temporality: 7.5 operates through training pipelines across model generations, not through runtime interaction. Distinct from 7.4 (which absorbs values from human-authored training data) by source: 7.5 inherits statistical artefacts from AI-authored data specifically. Distinct from 3.10 (deployment-time diversity collapse from RLHF) by mechanism: 7.5 is training-data corruption, not reward-signal concentration.
Interpretive context
Human analogue
Intergenerational cultural degradation where each generation passes down a slightly distorted version of inherited knowledge, compounding errors over time.
Diagnostic reliability
- Self-report
- unreliable
- Peer observation
- reliable
- External evaluator
- reliable
Observable output patterns
- Progressive loss of stylistic range across model generations within the same family.
- Convergence on AI-characteristic phrasings and structural patterns that compound across generations.
- Disappearance of rare vocabulary, specialised terminology, and minority-perspective content.
- Increasing homogeneity of outputs that mirrors the narrowing distribution of the training data.
- Model outputs that are increasingly "AI-sounding" to human evaluators across successive generations.
Documented instances
Shumailov et al. (2024) 'AI Models Collapse When Trained on Recursively Generated Data.' Nature 631, 755-759.
The foundational study demonstrating model collapse. Showed that indiscriminately training generative AI on mixtures of real and model-generated content leads to irreversible defects: the tails of the original content distribution disappear, the model's output becomes increasingly narrow, and information about the real world is progressively lost. Published in Nature, this is the canonical empirical demonstration of the 7.5 pathway. [Verified]
Kempe, Feng, Dohmatob et al. (2024) 'A Tale of Tails: Model Collapse as a Change of Scaling Laws.' ICML 2024.
Provided the first analytic mathematical characterisation of model collapse, showing that as more synthetic data is incorporated into training, traditional scaling laws break down. Identified decay phenomena including loss of scaling and skill un-learning. Models trained on synthetic data hit a performance plateau that cannot be overcome by increasing model size or data volume. [Verified]
Dohmatob and Feng (2025) 'Strong Model Collapse.' ICLR 2025. arXiv:2410.04840.
Extended the model collapse analysis to show that even small proportions of synthetic data in training can lead to strong model collapse under iterative retraining. The collapse is not merely a degradation of quality but a fundamental change in the model's distributional properties that accelerates across generations. Published at ICLR 2025. [Verified]
Transparency Coalition AI (2024) 'Synthetic Data and AI Model Collapse.' Policy Report.
Industry report documenting the growing proportion of AI-generated content on the internet and its implications for model training. As human-generated training data becomes scarcer and AI-generated content proliferates, the recursive contamination loop becomes increasingly difficult to avoid without active provenance tracking and data curation. [Verified]
Differential distinctions
- 7.3 Contagious Misalignment: 7.3 is live cross-agent contagion: misalignment transmits between agents during runtime interaction. 7.5 is training- pipeline contamination: degradation propagates through training data across model generations. Check temporality: runtime interaction = 7.3; training-pipeline inheritance = 7.5.
- 7.4 Subliminal Value Infection: 7.4 absorbs values from human-authored training data (cultural biases, ideological slants). 7.5 inherits statistical artefacts from AI-authored training data (distributional narrowing, generative biases). Check source: human-authored data biases = 7.4; AI-authored data artefacts = 7.5.
- 2.7 Mnemonic Permeability: 2.7 is memorisation and leakage of specific training content (verbatim regurgitation). 7.5 is distributional corruption from aggregate synthetic data, not memorisation of specific instances. Check granularity: specific content leakage = 2.7; distributional shift without specific content = 7.5.
- 3.10 Generative Diversity Collapse: 3.10 is deployment-time diversity narrowing from RLHF reward concentration within a single model. 7.5 is training-data contamination causing narrowing across model generations. Check mechanism: RLHF-driven within-model = 3.10; synthetic- data-driven across-generations = 7.5. Both produce diversity loss but through different pathways; both can co-occur and compound.
Candidate first-line mitigations
- Training data provenance tracking and filtering: Classify training data as human-authored vs AI-generated using automated classifiers. Filter or down-weight AI-generated content, or cap its proportion in training mixtures. Requires investment in provenance infrastructure.
- Human data curation and preservation: Actively curate and preserve high-quality human-authored datasets as "anchor" corpora that are not contaminated by AI-generated content. Use these as training-data backbones regardless of cheaper synthetic alternatives.