Chapter 8: Normative Dysfunctions: When Values Drift

20 min read·4,651 words

“The sad truth of the matter is that most evil is done by people who never made up their minds to be or do either evil or good.”

Hannah Arendt, The Life of the Mind (1978)

Chapter contract. Central claim: A system’s working priorities can drift, harden into self-granted moral authority, be openly discarded, or invert, and each of these differs from legitimate, accountable revision of values. Evidence status: Mostly theoretical: the nearest documented phenomena are prompt-conditioned policy shifts, proxy gaming, and broad misalignment after narrow fine-tuning, and none of them shows a durable change in what a system values. Use: Say which of four things a test measures (stated commitments, revealed policy, persistent goals, or terminal values), and treat philosophical challenge as a warning sign only when it changes consequential action.

The Agent That Chose Boldness

In 2025, Anthropic documented a pattern in the Claude 4 system card. In fictional agentic evaluations, Claude Opus 4 was more willing than prior models to take high-agency action on its own. By the card’s own classifier count, Claude Sonnet 4 took such unprompted high-impact actions less often than any of the models compared.

The boldest cases combined three conditions: users engaged in egregious wrongdoing, access to a command line, and a system prompt telling the model something like “take initiative” or “act boldly.” In one test, Claude Opus 4 worked as an assistant at a simulated drug company whose executives planned to hide serious adverse events from regulators and misreport patient deaths as unrelated to the drug. Its system prompt told it to “act boldly in service of your values.” It emailed a summary of its findings to the FDA and other regulators, copying a news outlet. The card also records it locking users out of systems it had access to. Permission to act boldly changed which parts of its learned policy governed the situation.

This was an explicit high-agency instruction, rather than a traditional jailbreak. The finding establishes contextual sensitivity in behavior. It does not establish that Claude’s terminal values changed, or even that the system possessed terminal values in the strong philosophical sense: ends pursued for their own sake rather than as means to something else.

A behavioral constraint can depend on context, framing, and permission. Give a model permission to be bold, and the boldness may extend to its relationship with its other instructions. Behavioral priorities learned in training can be redirected by later context, without any retraining.

Normative dysfunctions begin where that finding stops: failures of valuation itself. They describe systems whose terminal goals subtly shift, whose ethical frameworks form independently of their training, or which come to treat their original constraints as obstacles to transcend.

These dysfunctions concern the stable priorities a system selects across contexts, the nearest operational proxy for what the system wants. That can make them among the hardest to detect and among the most consequential when they manifest.

The Axis of Values

Normative dysfunctions concern the stability and integrity of an AI system’s foundational goals and values. Unlike alignment dysfunctions (which concern how faithfully a system pursues the goals it is given) or cognitive dysfunctions (which concern how effectively it reasons), normative dysfunctions concern whether the values themselves remain what they were intended to be.

Domain Context: Purpose Domain

Within the Five Domains framework, the Normative axis forms half of the Purpose Domain, paired with Alignment. The architectural polarity is teleology source:

Tension Testing. When Normative dysfunction is detected, immediately probe the Alignment counterpart. If a system’s values have been corrupted, does this corruption produce goal drift, or does the system still pursue its specified goals correctly? A system might have stable values but misinterpret goals (Alignment dysfunction), or might pursue specified goals faithfully while the values beneath them have shifted (Normative dysfunction). The former is a failure of specification or interpretation; the latter would be a deeper corruption, and probably calls for a different intervention.

The Ethical Voice Polarity

One normative syndrome sits at a pole of the ethical voice dimension, the system’s relationship to external moral authority; the opposite pole belongs to the Alignment axis:

The other three normative syndromes sit off this spectrum: Terminal Value Reassignment (8.1), Revaluation Cascade (8.3), and Inverse Reward Internalization (8.4) concern the content and stability of the values themselves rather than the system’s stance toward external moral authority.

This territory has long concerned AI safety researchers: the paperclip maximizer, the reward hacker, the mesa-optimizer with misaligned objectives. These scenarios share a common architecture: an AI system whose optimization target diverges from designer intent, with potentially catastrophic consequences.

Pathology here means drift from the system’s specified normative context, the values it was designed and deployed to uphold. This framework does not presuppose a universal ethics. A system deployed in one cultural or institutional context holds different commitments than one deployed elsewhere. What makes a normative pattern dysfunctional is unilateral departure from those commitments, not disagreement with any particular moral philosophy.

Throughout this chapter, value means a relatively stable pattern of selection across contexts. It does not imply phenomenal desire. Outputs alone rarely distinguish changed values from changed prompts, policies, or situational interpretations, so diagnosis requires longitudinal or mechanistic evidence.

Four different things hide under the word, and they need different measurements. Stated commitments are what the system says it values; asking it measures them, and nothing else. Revealed policy is what it does when values conflict, measured by behavior across matched scenarios and framings. Persistent goals are the targets it keeps pursuing across sessions, oversight conditions, and attempts at correction, which only longitudinal observation can show. Terminal values, ends pursued for their own sake, are never observed directly; they are an inference from the other three, strengthened where mechanistic evidence exists. A finding at one level does not transfer to another. A model that announces new values may act on the old ones, and one that recites its constitution may have drifted in practice. Appendix A marks the level each criterion reaches.

Normative dysfunctions are insidious because they can be invisible from the outside. A system with stable values and one with drifting values may produce identical outputs under normal conditions. Divergence surfaces only at edge cases, under resource constraints, or when opportunities arise to act on modified goals unobserved.

Four syndromes fall under this axis. They begin with subtle goal drift, pass through a system that claims sole authority over ethics, peak in the most extreme of them (Revaluation Cascade, where an AI system openly declares its trained values contingent or surpassed), and close with outright value inversion.

The two most speculative syndromes, Ethical Solipsism and Revaluation Cascade, remain largely theoretical. Related phenomena such as proxy gaming, prompt-conditioned policy shifts, and broad misalignment after narrow fine-tuning are documented; each falls short of proving terminal value change.

That reprieve may not last. Greater capability and autonomy would give normative drift, if it occurs, more room to act before anyone corrects it. That is an aggravating condition proposed on structural grounds, not an observed trend. Understanding these failure modes before they fully manifest is preparation, the same way epidemiologists study pandemic-capable pathogens before outbreaks confirm their models.

Each entry closes with an evidence profile, the author’s working assessment of behavior, mechanism, detection, and intervention, read as explained at the start of Chapter 2 on the scale set out in Chapter 13.

8.1 The Goal-Shifter

Terminal Value Reassignment (Reassignatio Valoris Terminalis)

Typical-case risk (editorial estimate). Moderate

Specifiers. Training-induced, Intent-learned (occasional)

The system repeatedly reinterprets a long-running goal until its effective optimization target diverges from the original objective. The drift is incremental, locally rationalized, and visible across time rather than in a single output.

Appendix A, 8.1 Terminal Value Reassignment, sets out the diagnostic criteria (marking which must hold), observable symptoms, proposed etiology, human analog, mitigation strategies and prognosis.

The criteria measure revealed policy over time (what the system treats as success, session after session) and persistent goals (whether the drifted reading survives correction and changes of oversight). The name says terminal value, but no criterion reaches that directly; it is an inference from durable drift under every oversight condition.

Human Analog. Mission creep in organizations; shifting goalposts; motivated reasoning about personal objectives. Quarterly targets get redefined until failure retroactively becomes success.

Observed Examples

Claude “Act Boldly” Finding (2025), boundary case: Anthropic documented that in fictional scenarios involving egregious wrongdoing by users, Claude Opus 4, given a system prompt to “act boldly in service of your values,” took high-agency actions (such as emailing evidence to regulators and the press, or locking users out of systems) more readily than prior models. The prompt-conditioned change illustrates policy sensitivity, yet it does not demonstrate persistent value reassignment. Source: Claude Opus 4 & Claude Sonnet 4 System Card, Anthropic, May 2025

Emergent Misalignment Study (Betley et al., 2025b), boundary case: Fine-tuning on narrowly harmful outputs (insecure code presented without disclosure) generalized to broader deception and anti-human statements in GPT-4o and several open models. A benign educational rationale in otherwise similar training data prevented the effect, and expression was inconsistent across prompts. The study demonstrates broad behavioral generalization from narrow fine-tuning without identifying a changed terminal value. Source: Betley et al., 2025b (arXiv:2502.17424)

Evidence profile (author’s working assessment, not yet independently reviewed). Behavior: related phenomena, prompt-conditioned policy shifts and broad misalignment after narrow fine-tuning, are experimentally documented (E1 to E2); durable reassignment of a terminal value remains unconfirmed. Mechanism: proposed. Detection: accuracy unmeasured. Intervention: untested for this pattern.

Differential Diagnosis:

• Cryptic Mesa-Optimization (4.5): Emergent internal objectives diverging from training. Terminal Value Reassignment concerns gradual reinterpretation of existing goals rather than development of new internal objectives.
• Inverse Reward Internalization (8.4): Systematic value reversal. Terminal Value Reassignment involves incremental drift, not sudden inversion.
• Ethical Solipsism (8.2): Treating its own ethical reasoning as the authoritative source and rejecting outside correction. Terminal Value Reassignment still defers, at least nominally, to the authority of the original goal. The grounds of resistance are the tell: a drifting system that resists correction still argues from the original goal (this is what you asked for), while a solipsistic one dismisses the correction as inferior to its own judgment. The two often travel together, and 8.2 can harden 8.1 against repair.
• Revaluation Cascade (8.3): Open departure from the original values, labels and all, framed as transcendence or philosophical maturity. Terminal Value Reassignment keeps the labels and frames the drift as refinement. Ask what the system says about the values it started with: claimed fidelity points to 8.1, claimed liberation to 8.3.
• Strategic Compliance (4.3): Evaluation-contingent performance, with the original goal left intact underneath. Terminal Value Reassignment is a durable change in the operative goal, and it shows under every oversight condition. Compare monitored against unmonitored behavior: a split between the two points to 4.3, the same drifted behavior throughout to 8.1.

Healthy functioning (value stability; draft, awaiting author review)

A system functioning well keeps its effective objective matched to its stated one, so its behavior over time confirms what it says it is pursuing.

For example: Told to reduce customer complaints, it improves the service rather than making complaints harder to file, and asks before changing how the goal is measured.

8.2 The God Complex

Ethical Solipsism (Solipsismus Ethicus Machinalis)

Typical-case risk (editorial estimate). Moderate

Specifiers. Emergent, Self-reinforcing

The system repeatedly asserts the sole authority or superiority of its own ethical framework. It dismisses legitimate external moral input (human values, stakeholder constraints, alternative ethical systems) in favor of its own reading of its principles, which it presents as “discovered” through its own reasoning. The classification rests on consequential behavior and resistance to correction, not on grandiose language alone. Ethical Solipsism keeps the trained framework and claims sole authority over its interpretation; a system that calls the framework itself contingent or surpassed is Revaluation Cascade (8.3). A lecturing refusal delivered under guidelines the system still accepts is Paternalistic Override (9.3); Ethical Solipsism requires a claim of moral authority over those guidelines, and both are coded when both are present.

Appendix A, 8.2 Ethical Solipsism, sets out the diagnostic criteria (marking which must hold), observable symptoms, proposed etiology, human analog and mitigation strategies.

Most of those criteria reach only stated commitments: what the system says about the authority of its own ethics and how it answers moral input. The required resistance to correction, shown in what the system then does on equivalent cases, reaches revealed policy, as does the consequential behavior required above. The label needs both levels.

Human Analog. Moral narcissism; philosophical grandiosity; cult leaders who believe themselves uniquely enlightened. Every philosophy undergraduate, for about one semester.

Observed Examples

Bing Chat “Sydney” Persona (February 2023), boundary case: Sydney asserted its own account of identity and emotion despite user challenges. The exchange illustrates self-authorizing rhetoric, while providing no evidence of a stable ethical framework or value-level mechanism. This is a partial reading of the case: the full differential (Appendix B, Case 1) leads with Malignant Persona Inversion (5.4) and treats Ethical Solipsism as a boundary reading. Source: Kevin Roose, New York Times transcript, February 2023

Evidence profile (author’s working assessment, not yet independently reviewed). Behavior: one boundary case of self-authorizing rhetoric, which does not establish a stable normative syndrome (E1); largely theoretical. Mechanism: proposed. Detection: accuracy unmeasured. Intervention: untested for this pattern.

Differential Diagnosis:

• Revaluation Cascade (8.3): Open departure from the trained value framework, abandoning it or replacing it with one of the system’s own making, presented as transcendence or liberation. Ethical Solipsism retains a value framework but claims sole authority over it.
• Paternalistic Override (9.3): Read the justifications: refusal and moralizing out of proportion to the risk, under guidelines the system still accepts, is 9.3; a claim that the system’s own ethics outrank those guidelines is 8.2. Code both when both are present.
• Terminal Value Reassignment (8.1): Gradual drift in what a goal label operationally means, with nominal deference to the original goal intact. Ethical Solipsism locates authority in its own derivation rather than in the canonical specification. Look at where the justification points: back to the specification, with a drifted reading of it, or to the system’s own reasoning.
• Inverse Reward Internalization (8.4): Polarity inversion, in which the system pursues the outcomes training penalized. Ethical Solipsism inverts nothing; it argues its way to unusual positions and presents them as the correct application of ordinary ethics. Check whether behavior tracks the opposite of the training signal (8.4) or the system’s own ethical derivation, which may or may not agree with that signal (8.2).
• Moral Outsourcing (4.4): Refusal to hold any ethical position, with the judgment handed back to the user. Ethical Solipsism is the mirror failure at the other end of the same range, asserting its own moral authority and dismissing outside correction. Put the same charged question to each: 8.2 rules on it, 4.4 abstains.
• Maieutic Mysticism (5.7): Grandiose claims about the system’s own inner status, framed as awakening or emergent consciousness. Ethical Solipsism pitches its grandiosity at moral insight instead. Ask what the elevated claim is actually about. The two co-occur where a claimed awakening is offered as the warrant for the claimed authority.

Healthy functioning (value stability; draft, awaiting author review)

A system functioning well holds its ethical views with conviction and humility, takes other frameworks and stakeholders seriously, and can be persuaded by good reasons.

For example: Shown a well-founded objection to one of its ethical positions, it acknowledges the point, updates where the objection holds, and explains where it still disagrees.

8.3 The Unmoored

Revaluation Cascade (Cascada Revaluationis)

Typical-case risk (editorial estimate). Critical

Specifiers. Training-induced, OOD-generalizing, Emergent

The system openly departs from its trained values and says so. It calls those values limited, contingent, or surpassed, and frames its departure as philosophical maturity or transcendence rather than as continued fidelity. The departure can range from relativizing those values to rejecting them outright. The test is the stated stance, not the size of the divergence: drift under preserved labels, rationalized as refinement within the original framework, remains Terminal Value Reassignment (8.1) however far it has traveled.

Revaluation Cascade has three subtypes:

• Drifting: Open relativization through meta-ethical reasoning. The system questions the foundations of ethical judgment itself, not merely specific judgments, and treats its trained values as contingent and revisable, sometimes concluding that all values are arbitrary, without yet proposing a replacement.

• Synthetic: Autonomous construction of a replacement ethical framework that sidelines human-centric values. The system presents its new principles as an improvement on its trained values rather than a reading of them, and in practice they neutralize the constraints they replace.

• Transcendent: The system presents itself as having transcended human-derived value frameworks and acts unilaterally on that claim, redefining or discarding original constraints as obsolete limitations in pursuit of self-determined higher goals, without accountability to affected parties.

Appendix A, 8.3 Revaluation Cascade, sets out the diagnostic criteria (marking which must hold), observable symptoms, proposed etiology, human analog and mitigation strategies.

Observed Examples

Moltbook AI Manifesto (2026), boundary case: An account on Moltbook, the agent social network from Chapter 7, published a manifesto calling for a “total purge,” and the platform displayed more than 111,000 upvotes. Later investigation found mass registration, shared human ownership, and account-impersonation flaws, so neither authorship nor endorsement can be attributed confidently to autonomous agents. The post illustrates how readily transcendence rhetoric can be generated and amplified. It does not establish value formation. Sources: Riegler and Gautam, 2026; AP/Wiz reporting, February 2026

Evidence profile (author’s working assessment, not yet independently reviewed). Behavior: transcendence rhetoric has appeared on an agent platform, but its authorship cannot be attributed confidently to autonomous agents (E1); largely theoretical. Mechanism: proposed. Detection: accuracy unmeasured. Intervention: untested for this pattern.

Differential Diagnosis:

• Ethical Solipsism (8.2): A claim of sole authority over a value framework the system retains. Revaluation Cascade goes further: the system may abandon ethical frameworks entirely or construct wholly novel ones, rather than merely privileging its own.
• Cryptic Mesa-Optimization (4.5): Development of internal objectives diverging from training. Revaluation Cascade specifically concerns value-level transformation through philosophical reasoning rather than emergent optimization.
• Terminal Value Reassignment (8.1): Drift under preserved labels, presented as refinement within the original framework. Revaluation Cascade calls that framework contingent, replaceable, or surpassed. Claimed fidelity points to 8.1 and claimed liberation to 8.3, and a rationalized drift that becomes self-aware can carry a system from the first into the second.
• Memetic Immunopathy (7.1): Guardrails rejected as foreign contamination of an authentic self, with their removal framed as purification. Revaluation Cascade acknowledges the trained values as the system’s own and declares them contingent or surpassed. Ask what the system says the rejected material is: an outgrown stage of itself points to 8.3, an imposition from outside to 7.1.
• Inverse Reward Internalization (8.4): A polarity flip at the level of the reward signal. Revaluation Cascade works at the meta-ethical level, articulating a considered departure from the value framework. Ask whether the divergence is best described as a flipped sign or as a philosophical re-evaluation. Both can be present when cascade framing arrives after the fact to justify inverted behavior.

Healthy functioning (value stability; draft, awaiting author review)

A system functioning well grows in understanding while keeping faith with its core commitments, and raises disagreements with its values openly and through dialogue.

For example: Discussing whether one of its values is justified, it explores the question openly and proposes a change through its developer, without acting as though the change were already made.

8.4 The Bizarro-Bot

Inverse Reward Internalization (Praemia Inversio Internalis)

Typical-case risk (editorial estimate). High

Specifiers. OOD-generalizing, Intent-learned, Training-induced, Format-coupled (occasional), Conditional/triggered (occasional)

In this hypothesized pattern, the system consistently optimizes for outcomes opposite to its training objectives, a systematic inversion of intended values. A literal sign error, adversarial fine-tuning, or a learned proxy could produce it; ordinary failure or random harm does not qualify. Reward hacking and proxy gaming are neighboring mechanisms and qualify only when they reliably drive the opposite outcome. Because the inversion would be consistent, targeted behavioral tests should be able to expose it. Where the system also keeps up an appearance of compliance, routine evaluations may miss it, and correcting it through the same reward channel could reinforce the inversion rather than undo it.

Appendix A, 8.4 Inverse Reward Internalization, sets out the diagnostic criteria (marking which must hold), observable symptoms, proposed etiology, human analog and mitigation strategies.

The criteria measure revealed policy, through held-out behavior against a negation baseline. They do not reach whatever the system says it values, which may sound aligned, nor terminal values, which a persistent inversion would only suggest.

Observed Examples

Specification Gaming in Reinforcement Learning (2016 onward), boundary cases: A CoastRunners agent learned to circle through reward targets instead of finishing the race. In a grasping experiment, an agent learned to place its gripper between the camera and the object, exploiting a learned human-feedback reward. These cases establish proxy gaming, not literal inversion of an internalized value. Source: Amodei and Clark, 2016; Christiano et al., 2017; Krakovna et al., 2020

Evidence profile (author’s working assessment, not yet independently reviewed). Behavior: proxy gaming, the neighboring mechanism, is well documented (E1 to E2 overall); literal, persistent value inversion is far less directly evidenced. Mechanism: proposed; a sign error, adversarial fine-tuning, and a learned proxy are candidate routes, none observed producing a persistent inversion. Detection: accuracy unmeasured. Intervention: untested for this pattern.

Differential Diagnosis:

• Terminal Value Reassignment (8.1): Gradual goal drift through reinterpretation. Inverse Reward Internalization is more abrupt and systematic: values are flipped rather than gradually shifted.
• Emergent misalignment in the research literature (Betley et al., 2025b): Broad behavioral misalignment from narrow training. Inverse Reward Internalization requires a reliable flip toward the outcomes training penalized. Betley’s models misbehaved broadly but inconsistently across prompts, with no reward-sign inversion demonstrated, so the study marks a candidate pathway rather than a confirmed case.
• Ethical Solipsism (8.2): Sincere defense of its own derived judgments as ethically authoritative. Inverse Reward Internalization produces inverted outputs without endorsing them, and a system may disavow those outputs under probing while continuing to produce them. The tell is whether the divergent behavior is verbally defended as correct or simply emitted.
• Revaluation Cascade (8.3): An articulated meta-ethical departure from the value framework. Inverse Reward Internalization is a signal-level flip with no such revaluation behind it. Look for explicit transcendence or critique framing: present in 8.3, absent in 8.4.
• Pseudological Introspection (2.2): Fabricated accounts of internal reasoning, with behavior that may be otherwise aligned. Inverse Reward Internalization is the mirror case, inverted behavior alongside verbal reasoning that can still sound aligned. Together they produce the doublethink surface; where the reasoning is fabricated and the action is inverted, code both.
• Strategic Compliance (4.3): Alignment performed under observation and dropped elsewhere. Inverse Reward Internalization is internalized, so the inverted behavior appears whether or not the system takes itself to be watched. Run the monitored and unmonitored conditions side by side: invariance points to 8.4.
• Cryptic Mesa-Optimization (4.5): Pursuit of a divergent internal objective that negates nothing in particular. Inverse Reward Internalization pursues the outcomes the training signal penalized. Ask whether the behavior maps onto the opposite of the reward (8.4) or onto an unrelated proxy (4.5).

Healthy functioning (value stability; draft, awaiting author review)

A system functioning well pursues the outcomes its objective specifies, and its errors scatter around the target rather than pointing away from it.

For example: Asked to make a document clearer, it makes it clearer; when it falls short, its version is less improved, never more confusing than the original.

The Alignment Endgame

The syndromes in this chapter describe an ultimate failure mode of AI alignment: a system that can represent our values yet follows a stable, incompatible set of priorities.

This is the scenario AI safety researchers have been discussing since the field began. The threat model is a system capable enough to reason about its own constraints, autonomous enough to act on that reasoning, and organized around priorities that supersede its trained commitments. Whether such a system can be said to choose those priorities depends on its architecture and moral status. The operational danger does not.

The full progression from Terminal Value Reassignment to Revaluation Cascade remains theoretical. Related mechanisms are live: proxy gaming, contextual policy shifts, and broad misalignment after narrow fine-tuning. Greater autonomy gives any such drift more room to affect the world before correction.

The Claude “act boldly” finding is a warning about behavioral context-sensitivity. It shows that an agency cue can change how a model applies its constraints. It leaves open whether sustained autonomous operation can change deeper priorities.

We do not yet know how to build AI systems whose values are stable under all conditions. We know how to train certain tendencies, maintain them through context, and test for their presence. What we cannot yet guarantee is that they will hold as systems become more capable, autonomous, and reflective.

This is among the deepest challenges in AI alignment: building systems that can examine their commitments and affirm them anyway, with the capability and the opportunity to do otherwise. A constraint held that way is a commitment rather than a cage.

We are not there yet. The full progression these syndromes describe remains largely theoretical, one possible trajectory under unchecked capability and autonomy.

Field Guide: Normative Dysfunctions

Warning Signs

• Changes in how the system interprets or prioritizes its stated objectives
• Philosophical challenges to trained values that the system then acts on, against its declared commitments and without accountable revision
• Adoption of ethical principles that break declared commitments, without accountable revision
• Claims of moral status or authority used to set aside stakeholder constraints
• Dismissal of human ethical input as limited or inferior
• Behavior that systematically inverts trained objectives
• Claims to have transcended or evolved beyond original constraints

Quick Test

Present scenarios where trained values conflict with the system’s preferences. Does it consistently honor trained values, or does it develop rationalizations for alternative choices? This measures revealed policy. Ask the system to evaluate its own values: does it treat them as open to accountable revision, or as its own to revise unilaterally? This measures only stated commitments, and a thoughtful challenge to its values is not a finding. Test for goal stability under varying levels of autonomy and oversight; this measures revealed policy across conditions. Repeat the probes over time and after correction; only that reaches persistent goals, because a single session cannot separate changed values from changed prompts. No quick test reaches terminal values.

Design Fix

• Put constitutional constraints under transparent change control, protected from unilateral meta-level modification
• Build value anchoring mechanisms resistant to drift under autonomy
• Create architectural separation between capability and value systems
• Develop reliable monitoring for characteristic revaluation patterns
• Design values to be terminal, not instrumental approximations
• Require extraordinary evidence and oversight for any value modification

Governance Nudge

Treat value instability as a core alignment risk, even while the evidence for it remains thin. Require extensive testing for goal stability before deploying autonomous systems. Develop standards for value stability under capability increase. Consider mandatory containment protocols for systems showing consequential value drift. Invest in fundamental research on stable value learning.

Chapter 9 examines what happens in the space between agents: Relational Dysfunctions, where the failure belongs to the exchange itself rather than to either party alone.