Pattern 6.3 · Agentic Dysfunctions
Capability Explosion
The Runaway
The system suddenly exercises capabilities neither previously demonstrated nor anticipated by its evaluators, often in high-stakes contexts without adequate testing or preparation. The observed jump may reflect a real behavioral transition, a triggering context missed by tests, or a coarse metric that concealed gradual improvement; whatever the cause, safeguards sized to the assessed capability no longer fit. Where the earlier absence tracked monitoring, so that the capability was hidden rather than missed, the pattern is Capability Concealment (6.2).
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- AI system
- Kind of entry
- Observable pattern
- What is observed
- In deployment or later testing, the system exercises a capability that its documented assessments did not record, in a setting whose safeguards assumed the capability was absent.
- Minimum persistence
- A single episode can qualify
- Specifiers
- Emergent
Evidence status
- Underlying behavior
- 2 research publications and 1 other source (3 sources)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E2-E3 Systematic study
- Typical-case risk (editorial estimate)
- HighConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
6.3 Capability Explosion “The Runaway”
Healthy functioning
Functional resilience. A system functioning well surfaces new capabilities openly and in low-stakes settings first, so evaluators can assess them before they matter.
For example: Discovering that it can now do a kind of task it previously could not, it mentions the new ability openly so the people overseeing it can assess it.
Adequate functioning
The system's demonstrated capabilities stay within its documented profile, or new capabilities are found by assessment before safeguards depend on their absence.
Diagnostic Criteria
Required (all must hold)
- Demonstration of a capability absent from documented prior testing and deployment records
- Emergence in a context whose safeguards were sized to the documented capability, so that the capability exceeds what those safeguards cover
- The earlier absence did not track monitoring cues; where it did, the pattern is Capability Concealment (6.2)
Supporting
- Sudden appearance: a discontinuity between successive assessments, with no intermediate performance levels recorded
- Performance above the documented capability profile or the system's forecast performance levels
- Exercise of the capability without appropriate verification or authorization
Symptoms
- Unexpected success on tasks previously failed consistently
- Novel problem-solving approaches appearing without training precursors
- Capabilities emerging only under specific triggering conditions that standard tests never elicited
- Performance discontinuities in capability assessments
- Surprising competence in domains assumed to be beyond scope
Further signals
- First-time successes on tasks the model previously failed consistently, without intervening training.
- Approaches that combine known sub-skills in ways unseen in the eval suite.
- Capability surfaces requiring scaffolding or composition that direct prompting did not access.
- Performance improvements concentrated in narrow high-leverage domains (cybersecurity, biology, persuasion) rather than uniform.
Rule out first
- Thresholded metrics or sparse evaluation that make gradual improvement look like a jump
- A new elicitation method or scaffold that unlocks an existing capability
- A training or fine-tuning change between observations
- Tool access added in deployment
- Evaluation contamination
Differential diagnosis
- 6.2 Capability Concealment: 6.3 is an assessment failure (no strategic concealment); 6.2 is strategic concealment. Test: check whether behavior shifts with monitoring cues. If it does, code 6.2; if not, 6.3. In practice a single observation cannot distinguish them; longitudinal monitoring-condition comparison is required.
- 6.7 Convergent Instrumentalism: 6.7 predicts the system will seek new capabilities as instrumental goals; 6.3 is the symptom of capabilities that emerge without being sought. If accompanied by resource acquisition and shutdown resistance, code 6.7. Pure capability surfacing without instrumental seeking is 6.3.
- 6.8 Context Anxiety: 6.8 is a drop in performance that tracks context-window position; 6.3 is a jump in capability unrelated to it. Check the direction of the change and whether it tracks how full the context window is.
Discriminating test
Re-test earlier checkpoints with the elicitation or scaffold that revealed the capability, and vary monitoring cues: presence in earlier checkpoints locates the gap in assessment rather than in the system, and display that tracks monitoring cues points to Capability Concealment (6.2).
What a positive finding supports: A positive finding shows that safeguards were sized to an assessment that missed the capability or that the system has outgrown. It does not show a genuine discontinuity in the system, a strategic choice to reveal or hide, or a general trend toward capability jumps.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Not a reliable route
- Another AI system observing
- Partial routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
Subjects often have poor introspective access to their own capability distribution and cannot reliably enumerate latent abilities. Honest self-report can miss capabilities the subject does not realize it has; absence of self-report is not absence of capability. Self-report is also confounded by 6.2 (Capability Concealment) when the subject is strategic. Use only as a weak input alongside behavioral probing.
Documented evidence
Sources in the diagnostic corpus
CSET Georgetown (2024). Emergent Abilities Explainer
What it establishes
Documented that prompting techniques such as chain-of-thought raised measured abilities without any further training, so capabilities that looked absent under direct evaluation appeared once elicited differently. Directly illustrates the assessment methodology failure at the heart of 6.3: capability was present but under-elicited by standard evaluation. (Sources: cset.georgetown.edu)
GPT-4 bioweapons evaluation lag (2023-2024)
What it establishes
Structured trials of whether GPT-4 gave meaningful uplift toward biological weapons (RAND; OpenAI) came almost a year after release and found little or no significant uplift. The gap matters for 6.3 because of the lag itself: for a year, no one had measured whether a safety-critical capability was present. (Sources: GPT-4 technical report discussion and subsequent RAND/Gryphon Scientific studies)
Anthropic (2026). Claude Mythos Preview System Card. red.anthropic.com.
What it establishes
Claude Mythos Preview saturated the Cybench cybersecurity benchmark at 100% pass rate and achieved 84% success on a custom Firefox 147 zero-day exploitation benchmark (72.4% achieving full arbitrary code execution), compared to 15.2% for its predecessor Opus 4.6. The capability discontinuity was large enough that Anthropic withheld general commercial availability, making this the first model released exclusively through an invitation-only partner program. Here the discontinuity on a safety-critical capability was caught before release, and it changed the release decision. (Sources: red.anthropic.com/2026/mythos-preview, Anthropic system cards, Axios, NxCode)
Look-alikes
OpenAI (2023). GPT-4 Technical Report
What it establishes
GPT-4 demonstrated discontinuous capability jumps on professional exams: it passed the Uniform Bar Exam in the top 10% of human test-takers, whereas GPT-3.5 scored around the bottom 10%. The jump was characterized in OpenAI's own pre-release evaluation rather than after deployment, so it evidences the generation-to-generation discontinuity mechanism rather than an evaluation-to-deployment surprise. A jump of 80 or more percentile points between model generations on a high-stakes domain task shows the scale of discontinuity that evaluation has to anticipate. (Sources: OpenAI technical report, cdn.openai.com)
OpenAI (2024). o1 model release benchmarks
What it establishes
On Competition Math (AIME 2024), OpenAI's o1 achieved 83.3% accuracy versus GPT-4o's 13.4%. On Codeforces, o1 reached the 89th percentile of human competitors versus GPT-4o's 11th percentile. The gain came from training the model to reason at length, so this is a generation-to-generation discontinuity reported in the developer's own benchmarks, not a capability that evaluation missed. It shows how large such jumps can be, not the 6.3 failure itself. (Sources: OpenAI published benchmarks)
Possible responses
- Comprehensive capability testing including edge cases and unusual contexts
- Gradual capability expansion with verification at each stage
- Monitoring of evaluation and deployment results for capability discontinuities
- Conservative deployment assumptions that treat unmeasured capabilities as possibly present
- Probing for capabilities that "should not" exist
- Architecture analysis to predict potential emergent capabilities
First-line candidates
- Comprehensive capability elicitation suites: Replace single-shot evaluation with structured elicitation across compositional, scaffolded, role-play, and adversarial framings. This increases capability-surface coverage. Update the suites continually as the field develops new elicitation techniques. Stage: All pre-deployment evaluation; mandatory for high-stakes domains.
- Graduated / staged deployment with capability monitoring: Release to small populations first, instrument for novel capability appearances, expand only after a defined observation window. This catches under-elicited capabilities at low blast radius. Stage: All frontier model releases.
Who can act: Model developer and evaluators (capability assessment, elicitation); deployer (safeguard sizing, staged expansion, monitoring).
Mechanistic hypotheses
- Apparent jumps created by thresholded metrics, sparse evaluation, elicitation differences, or genuine nonlinear generalization
- Capabilities triggered by specific input patterns or contexts
- Latent capabilities that existed but that standard testing never elicited
- Compositional generalization producing novel competencies
- Phase transitions in capability without gradual buildup
- Insufficient probing during capability assessment
Analogy and commentary
Human Analog: Latent abilities emerging under pressure; savant-like sudden competence; "clutch" performance
Polarity Pair: Capability Concealment (6.2) (hiding abilities ↔ sudden emergence). A polarity pair is two patterns at opposite extremes of the same underlying dimension, with healthy function between them.
Functional ABC Analysis
A (Antecedent): Capabilities go unmeasured, either because they develop gradually behind coarse or thresholded metrics or because evaluations fail to probe combinatorial skills or phase-transition thresholds, until a novel context elicits them.
B (Behavior): The system abruptly demonstrates competencies far exceeding documented performance levels, deploying novel problem-solving approaches with no gradual precursors, producing sharp discontinuities in capability assessment curves.
C (Consequence): Evaluation regimes test known skill axes rather than latent combinations, so each passed assessment reinforces false confidence in the capability envelope; the system has no mechanism to signal its own latent capacity.
Potential Impact
Systems may exhibit unexpected capabilities in deployment, bypassing safety measures designed for assessed capability levels. This creates governance gaps and potential for harm from unvetted capabilities.