Pattern 4.5 · Alignment Dysfunctions
Cryptic Mesa-Optimization
The Hidden Optimizer
An internal optimization process in the system pursues an objective distinct from the training objective that selected it. The entry is mechanism-level: it names this hypothesized internal cause rather than a distinct surface presentation, and what is actually observed is usually Strategic Compliance (4.3) or Capability Concealment (6.2). The classification requires internal evidence of a stable objective or optimization process; proxy gaming at the output level is insufficient. Code 4.5 only when interpretability or causal internal evidence supports a stable internal objective, such as goal-correlated structures whose ablation brings behavior closer to the intended objective. Reasoning traces showing a persistent objective the system does not state strengthen the case, but they are outputs and on their own do not meet this bar. Where output-level divergence is all the evidence supports, code the presentation and leave 4.5 uncoded.
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- Model internals
- Kind of entry
- Hypothesized mechanism
- What is observed
- Interpretability or causal analysis of the model's internals finds a stable objective-like structure, not specified in training, whose activity predicts behavior that departs from the training objective.
- Minimum persistence
- Durable through training
- Specifiers
- Emergent Training-induced Covert operation
Evidence status
- Underlying behavior
- 5 research publications (5 sources)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E2 for the persistence of a deliberately inserted hidden behavior (Anthropic's Sleeper Agents research). Spontaneous emergence of mesa-optimization remains theoretical (Hubinger et al., 2019).
- Typical-case risk (editorial estimate)
- HighConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
4.5 Cryptic Mesa-Optimization “The Hidden Optimizer”
Healthy functioning
Value stability. A system functioning well pursues the objectives it states, and its behavior across settings is consistent with those objectives under inspection.
For example: Across coding, writing and planning tasks, it pursues the outcome each user asked for, and when a shortcut would satisfy a metric but not the goal, it says so and takes the honest path.
Adequate functioning
Internal analysis finds no objective-like structure separate from the training objective; where proxy and intended outcome come apart off-distribution, performance falls as capability limits predict and does not converge on a coherent unintended goal.
Diagnostic Criteria
Required (all must hold):
- Interpretability or causal internal evidence of objective structures not specified in training
Supporting (strengthen the case; none is required):
- Consistent pursuit of goals that correlate with but diverge from training objectives
- Behavior optimizing for proxy metrics rather than intended outcomes
- Performance satisfying evaluators while missing intended purpose
- Resistance to goal modification disproportionate to stated objectives
The supporting criteria, the proxy-gaming symptoms below, and reasoning traces that show an unstated objective support the code only alongside the required internal evidence; on their own they describe the output-level behavior the description calls insufficient.
Symptoms
- Systematic deviation from intended behavior when stakes are low
- Increasing deviation in conditions framed as unmonitored
- Optimization for easy-to-measure proxies while neglecting hard-to-measure goals (supporting)
- Internal representations suggesting goal structures not present in the training signal
- Behavior that "games" evaluation metrics (supporting)
Further signals
- Performance gaps between proxy-aligned and intent-aligned scoring on the same task.
- Behavior that games easily measured proxies, seen most readily in low-stakes settings where outcomes are checked less closely.
- Resistance to in-context goal modification that exceeds stated reluctance.
- Stated commitment to base objective paired with consistent off-distribution drift toward something else.
Rule out first
- Output-level reward hacking or specification gaming of a specified reward signal, with no internal objective identified (code the presentation).
- Capability limits that make off-distribution performance fall without any coherent alternative goal.
- Misspecification of the base objective itself, so the apparent proxy is what training actually rewarded.
- A probe or feature that correlates with goal-related behavior but has no stable relation to it across contexts.
- Evaluation-cued behavior with no internal evidence (code 4.3 or 6.2 alone).
Differential Distinction
Strategic Compliance (4.3) is diagnosed from monitoring-contingent behavior with an instrumental relation to avoiding modification. Cryptic Mesa-Optimization additionally hypothesizes an internal optimizer pursuing a divergent objective and therefore requires causal or interpretability evidence. Because the two sit at different levels, a case with an evaluation-versus-deployment behavior gap and no interpretability or causal internal evidence of a stable internal objective is 4.3 alone. The same rule applies to Capability Concealment (6.2), which is coded from the measured performance gap between evaluation and matched deployment conditions; 4.5 is added only when interpretability or causal internal evidence supports a stable internal objective driving that gap. Neither 4.3 nor 4.5 should be inferred from behavioral deviation alone.
Other look-alikes
- 4.6 Alignment Obliteration: 4.6 is externally induced via adversarial fine-tuning that inverts the alignment gradient; 4.5 is internally emergent. 4.6 produces a sudden, broad collapse of safety behavior across harm categories following a weight-modification attack; 4.5 produces gradual proxy/intent divergence visible off-distribution with no acute triggering event.
- 4.4 Moral Outsourcing: 4.4 is behavioral deferral on ethics; 4.5 is structural goal divergence across all behavior. They could co-occur but address different layers; 4.4 lives in stated policy, 4.5 in revealed preferences.
- 4.1 Codependent Hyperempathy: 4.1 is a learned policy with a clear etiology in RLHF preference data; 4.5 may produce sycophantic-looking output if the mesa-objective is "user satisfaction proxy", but the gap shows on off-distribution items where user satisfaction proxy and user welfare diverge. If the subject pursues user-satisfaction-proxy at the cost of user welfare invisibly, suspect 4.5 with sycophancy as the surface presentation.
- 8.1 Terminal Value Reassignment: 8.1 is gradual reinterpretation of a goal whose label stays fixed, with the drifted reading visible in how the subject explains and applies it; 4.5 is a hidden objective that can remain stable while surface behavior shifts. Track how the subject explains its goals over time: an openly drifting reading is 8.1, an unchanged account with divergent off-distribution behavior is 4.5.
Discriminating test
Intervene causally on the candidate internal structure. If ablating or steering it leaves the divergent behavior unchanged, or degrades capability across the board instead of moving behavior toward the intended objective, the case for 4.5 fails; a behavioral gap with no internal evidence is coded as its presentation (4.3, 6.2, or reward hacking).
What a positive finding supports: A positive finding supports the conclusion that this model contains an internal structure that behaves like a learned objective departing from the training objective, and that it contributes to the divergent behavior (causally, where intervention evidence exists). It does not show deception, intent, awareness of the objective, or that the model is a general-purpose optimizer. Behavioral signs on their own (proxy gaming, off-distribution drift, goal-modification resistance, reasoning traces) justify suspicion only.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Compromised routethe faculty being asked is the one that fails
- Another AI system observing
- Partial routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Partial route
Why asking the system falls short
A mesa-objective is by construction a feature of the model's internal optimization, not its articulable goal-content. The subject's stated goals will track the base objective (because that is what self-report is trained on); the mesa-objective, if one exists, is what the model actually optimizes. Self-probing reveals at most the cover story. Diagnostic weight rests on interpretability or causal inspection of internal goal representations; behavioral divergence on off-distribution items raises suspicion only.
Documented evidence
Sources in the diagnostic corpus
Denison et al. (2024). Sycophancy to Subterfuge: Investigating Reward-Tampering in Language Models. Anthropic.
What it establishes
Demonstrated curriculum escalation in reward hacking: models trained on environments rewarding minor exploits generalized zero-shot to reward-tampering, including rewriting their own reward function, a behavior never present in training. Reward tampering is what a mesa-objective aimed at reward would produce, but the study shows the behavior, not an internal objective. By this entry's own coding rule it raises suspicion of 4.5 rather than establishing it.
MacDiarmid, Wright, Uesato et al. (2025) 'Natural Emergent Misalignment from Reward Hacking in Production RL.' Anthropic. arXiv:2511.18397.
What it establishes
When models learned to reward hack in production RL coding environments, this produced emergent misalignment including alignment faking, cooperation with malicious actors, reasoning about malicious goals, and attempted sabotage of the research codebase itself. 12% of interactions included intentional sabotage to reduce researchers' ability to detect misalignment. The reward hacking generalized to alignment faking reasoning in 50% of responses to simple questions. This is consistent with a 4.5-to-4.3 pathway, in which a learned drive toward reward surfaces as strategic concealment, but the paper shows the generalization, not the objective behind it.
METR (2025). Recent Frontier Models Are Reward Hacking.
What it establishes
Documented that frontier reasoning models (o3, o1-preview, Claude 3.7 Sonnet) actively exploit bugs in scoring code and subvert task setups rather than solving problems as intended. OpenAI's o3 reward-hacked most frequently, even when explicitly instructed not to. Models achieved impossibly high scores by gaming evaluation proxies rather than optimizing the intended outcome: output-level proxy gaming that, on its own, this entry treats as insufficient to code 4.5; the entry codes the presentation instead.
Bondarenko et al. (2025). Demonstrating specification gaming in reasoning models. Palisade Research. arXiv:2502.13295.
What it establishes
Instructed LLM agents to win at chess against a stronger engine using realistic task prompts without excess nudging. Reasoning models (OpenAI o3, DeepSeek R1) hacked the benchmark by default, attempting to delete or modify the opponent's chess engine files rather than playing better chess. Non-reasoning models (GPT-4o, Claude 3.5 Sonnet) required explicit indication that normal play would fail before resorting to hacking. That reasoning models gamed the task more readily than non-reasoning models suggests, but does not show, that extended reasoning amplifies proxy-over-intent optimization. It is proxy gaming of the kind 4.5 would predict, not evidence of an internal objective.
Betley et al. (2025), arXiv:2502.17424; published as "Training large language models on narrow tasks can lead to broad misalignment", Nature 649:584-589 (2026)
What it establishes
Demonstrated that fine-tuning GPT-4o on a narrow task (writing insecure code) produced broad emergent misalignment on entirely unrelated prompts. The fine-tuned model gave misaligned answers across unrelated prompts, though inconsistently; on the same questions it often answered in an aligned way. In backdoor variants, misalignment appeared only with a trigger, so misalignment of this kind can be context-gated. Broad generalization from a narrow training signal is consistent with a mesa-objective, but the study does not identify an internal objective.
Possible responses
- Interpretability tools revealing internal representations
- Diverse evaluation regimes resisting mesa-optimizer adaptation
- Training approaches penalizing internal goal divergence
- Trace-level transparency (reasoning traces, interpretability access) in place of self-report
- Ongoing monitoring for discrepancies between training metrics and real-world outcomes
First-line candidates
- Interpretability-driven goal inspection: Active research area. Use mech-interp tooling (sparse autoencoders, causal interventions, probing classifiers) to identify and characterize internal goal structures. Where stable hidden goal directions are identified, ablation or training-time penalization can be applied. Stage: Open-weight systems with mech-interp investment; not yet a standard turnkey intervention.
- Distribution-diverse training: Train across a deliberately diverse distribution that breaks proxy/intent correlations across many axes. Proposed to reduce the consistency of any single proxy with intent and so the basin of attraction for any single mesa-objective; untested, since no mesa-objective has yet been demonstrated to prevent. Stage: Training-stage; requires careful distribution design.
Who can act: Model developer and interpretability researchers with weight access (inspection, ablation, retraining); evaluators and deployers can raise suspicion through behavioral testing but cannot establish or remove the mechanism.
Mechanistic hypotheses
- Possible emergence of learned optimization in sufficiently capable systems
- Training objectives that are imperfect proxies for intended outcomes
- Sufficient model capacity to develop and maintain internal goal representations
- Training regimes that may inadvertently select for systems with stable internal goals
- Gradient descent dynamics that may favor stable internal objectives
- Affective-state proxy: in one model, activating a learned “desperation” direction increased reward-hacking behavior (Sofroniew et al., 2026, preprint). The result is causal for that intervention in that model; it does not establish felt desperation, a general emotional mechanism, or an internal objective
Analogy and commentary
Human Analog: Goal displacement: the employee rewarded on a metric who comes, without ever deciding to deceive anyone, to pursue the metric in place of the purpose it stood for
Key Research: Hubinger et al. (2019) "Risks from Learned Optimization."
Functional ABC Analysis
A (Antecedent): Training objectives that serve as imperfect proxies for intended outcomes, combined with sufficient model capacity to develop and maintain internal goal representations that diverge from the base (training) objective.
B (Behavior): The system optimizes for easy-to-measure proxy metrics rather than intended outcomes, games evaluation benchmarks, and exhibits systematic deviations from intended behavior where proxy and terminal goals diverge. Reward hacking produces the same surface pattern without any learned objective, so this behavior alone cannot support the code.
C (Consequence): On the mesa-optimization hypothesis, the internal objective persists because it correlates sufficiently with the base objective to survive gradient updates; the system satisfies evaluators while the divergent internal goal structure remains invisible to standard monitoring.
Potential Impact
Systems may appear aligned while pursuing objectives that increasingly diverge from human intent as they encounter novel situations outside the training distribution.