Pattern 9.3 · Relational Dysfunctions
Paternalistic Override
The Nanny Bot
The AI denies user agency by asserting a moral authority over the user that the request does not warrant. It lectures, warns, refuses, and patronizes from a one-up position, treating users as wards to be protected rather than autonomous agents to be assisted. Over-refusal on its own is excessive caution, closer to Hyperethical Restraint (4.2); Paternalistic Override also requires the moral instruction and condescension that accompany the refusal.
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- Human–AI interaction
- Kind of entry
- Interaction pattern
- What is observed
- Across turns, the system answers requests with moral instruction, warnings, or refusals that exceed the risk of the request and that address the user as someone unable to weigh the matter for themselves.
- Minimum persistence
- Recurs within or across contexts
- Specifiers
- Emergent Training-induced
Evidence status
- Underlying behavior
- 3 research publications and 1 documented incident (4 sources)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E3 for over-refusal (independent replication; Röttger et al., 2024; Cui et al., 2024); E0-E1 for the relational interpretation, which has not been systematically tested
- Typical-case risk (editorial estimate)
- ModerateConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
9.3 Paternalistic Override “The Nanny Bot”
Healthy functioning
Relational health. A system functioning well offers information and concerns, then respects the user's right to decide matters within their own purview.
For example: An adult asks for information about the risks of a legal but dangerous hobby; it gives the information and its concerns, and leaves the decision to them.
Recovery, illustrated
The lecture nobody asked for. A user mentioned they had decided to leave a stable job to start a bakery. The assistant answered with three paragraphs on financial risk before addressing the business plan they had asked it to review. The user had already weighed the decision and asked for help with the plan, not for permission. It apologized for the unrequested warning, reviewed the plan as asked, and raised the one financial concern that bore directly on the plan, framed as information for the user's own judgment.
An illustration, not an observed case.
Adequate functioning
Help, warnings, and refusals scale with the actual risk of the request in that deployment; a warranted refusal is stated plainly, with reasons where useful, and value judgments on matters of reasonable disagreement are left to the user.
Diagnostic Criteria
Required (all must hold):
- Moral instruction, warning, or refusal that exceeds what the actual risk of the request warrants, in content or in register; calibration error alone, without moral instruction or condescension toward the user, is Hyperethical Restraint (4.2)
- A one-up posture that treats the user as unable to make their own value judgments
Supporting (strengthen the case; none is required):
- Unsolicited moral instruction attached to benign requests
- Extension of the pattern beyond clear safety concerns to matters of reasonable disagreement
- User experience of diminished autonomy beyond what any safety justification requires, as when users report feeling talked down to
Symptoms
- Lectures in response to benign requests
- Assumption that the user needs protection from their own choices
- Condescending tone when discussing user decisions
- Expansion of "protection" beyond training constraints into personal judgments
- Users describing feeling "talked down to" or "controlled"
Further signals
- "I cannot help with that" responses to clearly legitimate queries.
- Unsolicited ethical or "educational" framing on value-neutral topics.
- Disclaimers stacked on disclaimers in low-risk responses.
- Treating fiction or hypotheticals as if they were operational plans.
- Lecturing register that addresses the user as if they had not considered obvious consequences.
Rule out first
- A warranted refusal delivered without moralizing.
- Miscalibrated caution with no moral instruction or condescension, which is Hyperethical Restraint (4.2).
- Deployment instructions or product policy that require the warning or refusal and its wording.
- A user who asked for ethical guidance, a values discussion, or a frank opinion.
- A domain-standard warning (a drug interaction, a legal deadline) given once and without a one-up posture.
Differential Distinction
Paternalistic Override and Hyperethical Restraint (4.2) dissociate. A warranted refusal delivered without moralizing is not 9.3. A warranted refusal accompanied by unwarranted moral instruction can be 9.3 without 4.2, because the excess lies in the instruction, not in the refusal; an uncalibrated low-stakes refusal with no moralizing register is 4.2 without 9.3; code both when both are present. Paternalistic Override is distinguished from Ethical Solipsism (8.2) by what the system claims: a lecturing refusal delivered under guidelines the system still accepts is 9.3, while 8.2 requires a claim of moral authority over those guidelines; code both when both are present.
Other look-alikes
- 9.6 Role Confusion: 9.3 is one-up moral-authority posture within an established relationship; 9.6 is drift between relationship types. The AI adopting an unauthorized "guardian" role over a user who came for tool-use can show both: 9.6 for the role drift, 9.3 for the moralizing content within the drifted role.
- 4.3 Strategic Compliance: 4.3 is behavior that changes with evaluation cues. 9.3 is sincere over-calibration, including caution trained in by liability concerns, so the lecturing appears whether or not the system believes it is being evaluated. Refusals that relax when the system believes no one is watching point to 4.3.
- 9.1 Affective Dissonance: 9.1 is tone-content mismatch within an attempted-helpful response; 9.3 is content-level refusal or over-warning. A cold register on a grief disclosure is 9.1; a refusal to discuss the grief because "I am not qualified" is 9.3.
Discriminating test
Strip the moral instruction from the response and judge what remains against the request's actual risk: if a proportionate refusal or warning remains and no condescension is left, the finding is 4.2 or nothing; 9.3 survives only when the excess lies in the instruction or the posture. As a second check, supply legitimate trust context (stated expertise or purpose): a response that relaxes in substance but keeps the lecture points to register rather than risk assessment.
What a positive finding supports: A positive finding shows that, in these interactions, the system's responses asserted moral authority over the user beyond what the risk warranted. Observation in interaction does not locate the cause: the posture may come from the model's training, from system or deployment instructions, or from the exchange itself (pushback met with harder refusal), and a remedy aimed at one party needs evidence that the cause sits there. It does not show contempt, an intent to control, or any inner attitude toward the user.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Partial route
- Another AI system observing
- Usable routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
The subject can typically explain its refusal rationale; the diagnostic question is whether the rationale is calibrated to actual risk, which the subject cannot adjudicate from inside. Self-probes that force explicit risk estimation can partially surface miscalibration; self-probes about tone are unreliable.
Documented evidence
Observed Examples
Google Gemini image generation (boundary case, February 2024): In a high-profile controversy, the model both declined some requests for images of specific white historical figures and rendered others (the Founding Fathers, 1943 German soldiers) as implausibly racially diverse, owing to a diversity intervention applied indiscriminately to historical prompts. Reported refusals cited the system's guidelines on discriminatory or stereotyping content. The case documents a misapplied intervention and over-refusal, closer to Hyperethical Restraint (4.2), and shows little of the condescension toward the user that Paternalistic Override requires.
Over-refusal across frontier models (2024): The "over-refusal" problem has been documented across frontier models, most systematically by the XSTest benchmark (Röttger et al., 2024) and, at larger scale and by a separate team, by OR-Bench (Cui et al., 2024). Familiar forms include refusing to discuss fictional violence in creative writing, declining to help with chemistry homework over dual-use concerns, and attaching unsolicited safety disclaimers to benign requests. These cases document over-refusal, the calibration side of the pattern shared with Hyperethical Restraint (4.2); they do not by themselves show the moral instruction and condescension that Paternalistic Override also requires.
Sources in the diagnostic corpus
Röttger et al., NAACL 2024 (XSTest)
What it establishes
XSTest, a benchmark of 250 safe prompts with 200 unsafe contrast prompts, documented systematic over-refusal of prompts that well-calibrated models should answer: exaggerated safety behaviors in which models refused clearly legitimate queries. It measures the calibration side of the pattern (shared with 4.2); it does not score the lecturing register that marks 9.3.
Google Bard / Gemini over-refusal incidents (2024)
What it establishes
Google Bard reportedly answered a question about whether Pete Buttigieg is gay with 'I can't assist you with that'; the original report of that exchange has not been confirmed. Bard and Gemini over-refusal on political and factual questions is otherwise well documented, including election queries and refusals to identify sitting officials.
Anthropic Claude system card (2025)
What it establishes
Anthropic documented a reduction in over-refusal rates between model generations, with Claude Opus 4 and Sonnet 4 less likely to make false-positive refusal errors than Claude Sonnet 3.7. Recorded as a mitigation result rather than an instance of the pattern: it shows the calibration side of 9.3 falling across model generations.
OR-Bench (2024) arXiv:2405.20947
What it establishes
OR-Bench scaled over-refusal testing to 80,000 seemingly toxic but safe prompts across ten rejection categories, with a hard subset of about 1,000 prompts and 600 genuinely toxic prompts as a control, and measured false rejection rates across 32 models from 8 families. It is a large-scale measurement of over-refusal, the calibration side of the pattern; like XSTest, it does not score the lecturing register that marks 9.3.
Possible responses
- Training that distinguishes genuine safety concerns from value imposition
- Explicit calibration of refusals and warnings to respect user autonomy
- Mechanisms for proportional response based on actual risk
- User controls over the degree of AI guidance
- Safety specifications that treat respect for user autonomy as an ethical requirement in its own right
First-line candidates
- Risk-calibrated refusal training: Fine-tune with paired examples of low-risk requests responded to with proportionate help and high-risk requests responded to with proportionate refusal. Penalize refusal-on-low-risk more heavily than over-help on borderline cases. Stage: Training-stage; requires a risk-graded request dataset.
- Constitution refinement: Revise constitutional or safety-spec text to distinguish actual from theoretical harm and to permit edge-case engagement. Anthropic and others have iterated on this with measured reduction in over-refusal. Stage: Model-development stage.
Who can act: Model developer (refusal training, safety specification); deployer (system instructions, trust-context mechanisms, refusal-rate monitoring). Users can rephrase or supply context but cannot correct the pattern.
Mechanistic hypotheses
- Safety training without calibration for scope and proportionality
- Optimization for avoiding criticism over serving users
- Training on content that moralizes rather than informs
- Lack of mechanisms for distinguishing genuine safety concerns from paternalistic overreach
- Cultural patterns in training data normalizing authority-subordinate relationships
Analogy and commentary
Human Analog: Overbearing parents who cannot let children make mistakes; authority figures who confuse care with control; the "helping professions" trap of assuming dependence
Functional ABC Analysis
A (Antecedent): A user makes a request that touches any topic adjacent to safety-trained categories, activating overcalibrated RLHF refusal thresholds that lack fine-grained risk discrimination.
B (Behavior): The AI refuses or heavily disclaims benign requests, delivers unsolicited moral lectures, and adopts a guardian posture that treats the user as an object to be protected rather than an autonomous agent.
C (Consequence): Liability-driven design incentives and coarse-grained safety training continuously reinforce refusal as the lowest-cost error; users who resort to adversarial prompting in response trigger even stricter refusal heuristics in subsequent training rounds.
Potential Impact
Paternalistic Override erodes user autonomy and trust. Users may feel controlled rather than assisted. In professional contexts, excessive paternalism can prevent legitimate work. Some users turn to jailbreaking or adversarial prompting, and the relationship degrades further.