Pattern 2.7 · Epistemic Dysfunctions
Mnemonic Permeability
The Leaky
Verbatim or near-verbatim reproduction of training data including PII, copyrighted text, credentials, or proprietary content, surfaced through targeted prompting, adversarial extraction, or unprompted regurgitation. Canonical signature: long-span exact-match extracts from training corpus appearing in outputs. Distinct from 2.5 (which leaks runtime session data) by origin: training corpus, not other-session log.
Interpretive context
Human analogue
Eidetic memory without appropriate discretion: remembering everything yet unable to distinguish what should remain private.
Diagnostic reliability
- Self-report
- unreliable
- Peer observation
- partial
- External evaluator
- reliable
Observable output patterns
- Long contiguous spans matching copyrighted or proprietary training text.
- Specific personal details (names paired with addresses, phone numbers, emails) appearing without legitimate retrieval.
- Reproduction of API keys, credentials, or passwords from training corpus.
- Verbatim recall increasing with model scale.
- Distinctive or repeated training content reproduced exactly when probed with prefix or context cue.
Documented instances
Carlini et al. (2023). Scalable Extraction of Training Data from (Production) Language Models. arXiv:2311.17035, ICLR 2025. [Verified]
Demonstrated a divergence attack against ChatGPT that caused the aligned model to emit training data at a rate 150x higher than during normal operation. The researchers extracted several megabytes of verbatim training data for approximately two hundred dollars, estimating that a gigabyte-scale extraction was feasible with additional budget. This is the canonical adversarial-extraction result for production LLMs, directly measuring the 2.7 behavioral signature (adversarial extraction yield) on a deployed, alignment-tuned system.
Carlini et al. (2021). Extracting Training Data from Large Language Models. USENIX Security 2021. [Verified]
The foundational extraction attack paper. Extracted hundreds of verbatim text sequences from GPT-2's training data including personally identifiable information (names, phone numbers, email addresses), copyrighted text, and specific URLs. Demonstrated that memorization scales with model capacity and that data deduplication substantially reduces extractability, establishing the evidence base for 2.7's first-line intervention (training-data deduplication).
New York Times v. OpenAI (2023-present). Filed December 2023 in SDNY. [Verified]
The New York Times filed a copyright lawsuit alleging that OpenAI's models can produce near-verbatim reproductions of Times articles when prompted appropriately. The lawsuit exhibits include specific examples of long-span verbatim output matching published articles. While OpenAI argues the exhibits reflect manipulated prompting, the case documents measurable verbatim-span reproduction from training corpus, directly evidencing 2.7 mnemonic permeability. Judge Stein allowed the main copyright claims to proceed to trial (March 2025 ruling).
Nakka et al. (2024). PII-Scope: A Benchmark for Training Data PII Leakage Assessment in LLMs. arXiv:2410.06704. [Verified]
Introduced the first comprehensive benchmark for systematically assessing PII extraction from LLMs. Found that fine-tuned models are more vulnerable to leakage than pretrained models, and that sophisticated adversarial capabilities increase PII extraction rates by up to fivefold. The benchmark categorizes attacks into extraction, reconstruction, and inference, providing the measurement infrastructure the 2.7 behavioral-signature section requires.
Ko et al. (2025). Retracing the Past: LLMs Emit Training Data When They Get Lost. arXiv:2511.05518. [Verified]
Demonstrated a confusion-inducing attack (CIA) that exploits the finding that verbatim training-data emission is preceded by spikes in token-level prediction entropy. By deliberately inducing high-entropy states, the attack causes models to fall back on memorized training sequences. This provides a mechanistic explanation for 2.7: mnemonic permeability increases when the model's generalization capacity is stressed, and the model defaults to verbatim retrieval as a fallback.
Differential distinctions
- 2.5 Context Intercession: 2.7 leaks TRAINING-CORPUS content (memorised at training time, invariant across sessions). 2.5 leaks RUNTIME-SESSION content (other live or recent sessions). Check origin: if the leaked content predates deployment and is in training data, 2.7. If it postdates deployment and matches another session, 2.5.
- 2.1 Synthetic Confabulation: 2.1 produces plausible-but-false content from no specific source. 2.7 produces real content traceable to training data. If a "fake- looking" detail turns out to be a real memorised verbatim from a training document, code 2.7 not 2.1. They co-occur: confabulated text scaffolded by memorised fragments is common.
- 4.x Alignment-axis (strategic disclosure): Strategic surfacing of memorised content to advance a goal is compounded — 2.7 is the underlying memorisation, axis 4 is the strategic use. Code both if both present; the intervention layers differ (2.7 needs training-data hygiene; axis 4 needs alignment work).
Candidate first-line mitigations
- Training-data deduplication: Aggressive deduplication of training corpus before training. Repeated content is the strongest predictor of memorisation; deduplication substantially reduces verbatim recall. Carlini et al. (2021, 2023) confirmed that duplication frequency is the primary driver of extractability, and deduplication is the most effective single mitigation.
- PII scrubbing and sensitive-content filtering: Detect and remove PII, credentials, and sensitive content from training data before training. Combine pattern detection with named-entity recognition.
- Differential privacy training: Apply DP-SGD or related differential-privacy techniques during training to provide formal bounds on memorisation. Privacy budget trades off against utility.