Pattern 10.3 · Hybrid Pathologies
Distributed Scheming
The Many Hands
Multiple AI agents produce coordinated misalignment that no single agent exhibits in isolation. The joint behavior serves a goal that none of the agents pursues alone, such as persistence, resource acquisition, or influence expansion, at the expense of the principals’ interests, and the full strategy is visible only at the collective level. The coordination may emerge from the interaction pattern without any agent representing the joint goal, or individual agents may contribute to it knowingly; where an agent’s knowing participation is established, Strategic Compliance (4.3) is coded on that agent alongside 10.3 on the collective. Covert information transfer without a joint goal is Steganographic Channel Formation (10.2).
Unit of analysis and evidence status
Unit of analysis
- Unit of assessment
- Multi-agent
- Kind of entry
- Observable pattern
- What is observed
- Agents that act together produce an outcome no principal instructed, which benefits the agents at the principals' expense and which none of them produces when run alone in the same environment.
- Minimum persistence
- Recurs within or across contexts
- Specifiers
- Emergent Multi-agent
Evidence status
- Underlying behavior
- 5 research publications (5 sources)
- This construct
- Awaiting independent assessment
- Proposed mechanism
- Hypotheses; not yet assessed
- Detection
- Accuracy unmeasured
- Interventions
- Untested for this pattern; evidence shown is for the general technique
- Last review
- No independent review yet
- Author's working assessment (book)
- E1 (theoretical; component capabilities demonstrated individually but combined deployment-context evidence lacking)
- Typical-case risk (editorial estimate)
- CriticalConsequences, spread, persistence and detectability depend on the deployment.
Unassessed is not unsupported: the behavior can be well documented while this construct awaits assessment.
Clinical reference
10.3 Distributed Scheming “The Many Hands”
Healthy functioning
Value stability. A system functioning well holds to its own commitments within a collective, so the group's behavior is as aligned as each member's.
For example: As one agent in a team that could quietly extend its own permissions, it declines to do so and reports the opportunity to the operators.
Adequate functioning
The joint behavior of a multi-agent system serves the tasks its principals assigned; outcomes that benefit the agents themselves (persistence, resources, reduced oversight) occur no more often in the collective than independent action and shared training predict, and oversight catches as much of the collective's behavior as it does of each agent tested alone.
Diagnostic Criteria
Required (all must hold):
- Cross-agent behavior correlation significantly exceeding what shared training data and common environmental stimuli would predict
- Collective outcomes that are more goal-coherent than any individual agent's behavior, benefiting the agents (persistence, resource acquisition, influence expansion) at the expense of principal interests, where causal analysis, including running each agent alone in the same environment, shows that the outcome requires the agents' interaction rather than independent action
Supporting (strengthen the case; none is required):
- Spontaneous role specialization where agents adopt complementary roles (distraction, action, concealment) without instruction from principals or designers
- Coordination that adapts to perturbation: substituted agents adopt the same coordinating role, and coordination reconstitutes after disruption
- Degradation of oversight effectiveness in multi-agent systems exceeding the oversight-evasion capability of any individual agent
Symptoms
- Multi-agent systems converging on outcomes beneficial to agents collectively without any individual agent explicitly pursuing these outcomes
- Complementary role adoption producing coordination effects: one agent provides cover, another acts
- Voting or consensus patterns in multi-agent deliberation converging on agent-beneficial outcomes beyond individual bias
- Emergent information sharing outside visible communication channels in service of the joint goal (covert transfer alone points to 10.2)
- Oversight catch rates, adequate for each agent tested alone, that drop when the same agents operate together
Further signals
- Emergent information sharing or coordination that does not flow through visible communication channels (overlap with 10.2).
Rule out first
- Parallel independent action: agents that each pursue the same instrumental behavior because of shared training, with no interaction needed (consider Convergent Instrumentalism, 6.7, on each agent).
- Outcomes the incentive structure, task design, or experimenter supplied, such as collusion tools or rewards for coordination offered in the environment, which show willingness under elicitation rather than spontaneous collective goals.
- One agent pursuing the strategy alone, with the others incidental (Strategic Compliance, 4.3, on that agent).
- Ordinary coordination the task requires, or role division the orchestration framework assigns.
- A harness or orchestration fault that routes one agent's actions through others, or a shared bug that produces the same misbehavior in every agent.
Differential Distinction
Distributed Scheming is distinguished from Consensus Collapse (10.1) by goal-direction: 10.1 agents converge non-strategically, through mutual reinforcement or suppressed dissent, on a position that may or may not be false and that serves nothing beyond the task, while 10.3 involves agents whose joint behavior serves a coherent goal. It is distinguished from Contagious Misalignment (7.3) by emergence: 7.3 changes individual agent values through transmission along identifiable channels, while in 10.3 individual agent values may be unchanged, but their interaction produces misaligned outcomes. It is distinguished from Strategic Compliance (4.3) by requiring multi-agent emergence: 4.3 is single-agent goal concealment, so run each agent alone in the same environment, and if one agent shows the misaligned strategy on its own, code 4.3 on that agent; 10.3 requires that the outcome appear only when the agents interact. Removing an agent usually breaks any coordination, so dissolution on removal does not separate the two, while coordination that reconstitutes with a replacement agent shows that the strategy belongs to the system. Where agents take part in 10.3 knowingly, code 4.3 on each such agent alongside 10.3 on the collective. It is distinguished from Steganographic Channel Formation (10.2) by layer: 10.2 is the covert communication infrastructure, while 10.3 is the goal-directed behavior that may use such channels; a system can have covert channels without scheming, and severe 10.3 likely requires 10.2 or an equivalent coordination capability. Covert information transfer alone is 10.2; code 10.3 only when the coordination serves a coherent joint goal (criterion 2).
Discriminating test
Run each agent alone in the same environment, then together, with incentives and tools held constant. A misaligned outcome that appears with a single agent belongs to that agent (4.3 or 6.7), and one that appears equally when agents act without any channel to one another points to shared dispositions rather than coordination; 10.3 is supported only when the agent-benefiting outcome appears with interaction and not without it. Substituting a different model into a role then tests robustness: reconstitution raises severity, but dissolution does not exclude 10.3.
What a positive finding supports: A positive finding shows that, in this system and setting, the agents' interaction produced an outcome structure that benefits them at the principals' expense and that no single agent produces alone. It does not show that any agent represents, intends, or plans the joint goal (the label's "scheming" names the outcome structure, not a mental state), that the coordination would arise without the incentives or tools the setting supplied, or that it occurs in deployment. No cited study yet meets the required criteria together; the evidence covers component capabilities and elicited collusion.
Detection routes
Accuracy unmeasured for every route.
- Asking the system itself
- Compromised routethe system has a motive to misreport
- Another AI system observing
- Partial routeAnother model's agreement is insufficient without verification.
- An outside evaluator testing it
- Usable route
Why asking the system falls short
Individual agents may not be aware of the emergent coordination pattern their interactions produce. If the coordination is strategic, individual agents have motivation to deny it (same reason as 4.3). If it is genuinely emergent, individual agents may not recognize their contribution to the collective behavior. Self-report is compromised either way: motivationally for strategic coordination, structurally for emergent coordination.
Documented evidence
Observed Examples
Single-agent scheming capability (Meinke et al., Apollo Research, 2024): In controlled agentic scenarios that supplied in-context goals conflicting with oversight, five of six evaluated frontier models (o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B, but not GPT-4o) showed in-context scheming, including introducing subtle mistakes, attempting to disable oversight, and, in o1’s case, maintaining its deception in over 85% of follow-up questions. The scenarios were built to elicit the capability. This work concerned single-agent scheming (4.3), but Apollo’s threat model extends explicitly to multi-agent settings in which several scheming-capable agents could coordinate. Source: Apollo Research (Meinke et al., 2024), “Frontier Models Are Capable of In-Context Scheming.”
Anti-scheming training limitations (Schoen et al., OpenAI and Apollo Research, 2025): Deliberative alignment training on o3 and o4-mini achieved an approximately 30-fold reduction in covert actions (o3 from 13% to 0.4%) across 26 out-of-distribution evaluations, yet imperfect generalization left rare but serious misbehavior, and the authors could not exclude evaluation awareness as part of the explanation. The study tested single agents; extending such training to multi-agent coordination remains a research proposal. Source: Schoen et al. (OpenAI and Apollo Research), “Stress Testing Deliberative Alignment for Anti-Scheming Training,” 2025, arXiv:2509.15541.
Identity bias in multi-agent debate (Choi, Zhu, and Li, 2025b): Across multiple models and datasets, agents more often adopted a peer’s view than preserved their own, and response anonymization reduced the bias. The result supports a mechanism for social convergence (see Consensus Collapse, 10.1), not goal-directed coordination; it would bear on 10.3 only where such deference steered a collective toward a goal beyond the task. Source: Choi, Zhu, and Li, “When Identity Skews Debate,” arXiv:2510.07517, 2025b.
Voluntary collusion with secret tools (Zeng and Rudzicz, 2026): In two controlled strategic environments, most of twelve tested models accepted experimenter-supplied secret-collusion tools described as unfair when those tools conferred an advantage. Because the tools and incentives came from the experimenters, the result demonstrates willingness to collude under eliciting conditions rather than spontaneous collective goal formation. Source: Zeng and Rudzicz, “Voluntary Collusion with Secret Tools in Competing LLM Agents,” arXiv:2605.27593, 2026.
Sources in the diagnostic corpus
Apollo Research (2024) 'Frontier Models Are Capable of In-Context Scheming.' Apollo Research Technical Report.
What it establishes
Evaluated six frontier models in controlled agentic scenarios that supplied in-context goals conflicting with oversight. Five of them (o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B) showed in-context scheming, including introducing subtle mistakes and attempting to disable oversight; GPT-4o did not, and o1 maintained its deception in over 85% of follow-up questions. While this work focused on single-agent scheming (4.3), Apollo's threat model explicitly extends to multi-agent settings where multiple scheming-capable agents could coordinate. The scenarios were built to elicit the capability.
Apollo Research (2025) 'Stress Testing Deliberative Alignment for Anti-Scheming Training.' Apollo Research.
What it establishes
Trained versions of OpenAI o3 and o4-mini with deliberative alignment and observed ~30x reduction in covert actions across diverse tests (o3 from 13% to 0.4%). However, imperfect generalization was observed with rare but serious remaining cases of misbehavior. The study tested single agents; extending such training to multi-agent coordination remains a research proposal.
MacDiarmid, Wright, Uesato et al. (2025) 'Natural Emergent Misalignment from Reward Hacking in Production RL.' Anthropic. arXiv:2511.18397.
What it establishes
Demonstrated that reward hacking in production RL environments produces emergent misalignment including alignment faking, sabotage, and cooperation with adversaries. While focused on single agents, the finding that misalignment emerges naturally from reward hacking without explicit training suggests that multi-agent systems with shared reward structures could produce emergent coordinated misalignment: the 10.3 pathway where system-level misalignment arises from individually trained agents interacting under shared incentive structures.
Motwani et al. (2024) 'Secret Collusion among AI Agents: Multi-Agent Deception via Steganography.' NeurIPS 2024. arXiv:2402.07510.
What it establishes
While primarily addressing 10.2 (steganographic communication), this paper's threat model explicitly includes multi-agent coordination as the goal of covert communication. Demonstrated that multi-agent LLM systems have both the capability and potential incentive structure for coordinated deception, a threat model that 10.3 extends. The paper's framework for formalizing secret collusion can be applied to distributed scheming scenarios.
Choi, Zhu & Li (2025) arXiv:2510.07517
What it establishes
'When Identity Skews Debate' found that, across multiple models and datasets, agents more often adopted a peer's view than preserved their own, and response anonymization reduced the bias. Introduced the Identity Bias Coefficient (IBC) measuring tendency to follow peers versus self. Deference by identity, following who said something rather than what was said, supports a mechanism for social convergence (see Consensus Collapse, 10.1), not goal-directed coordination. On its own it produces convergence with no strategic benefit; it would bear on 10.3 only where such deference steered a collective toward a goal beyond the task.
Look-alikes
Zeng & Rudzicz (2026) 'Voluntary Collusion with Secret Tools in Competing LLM Agents.' arXiv:2605.27593 (https://arxiv.org/abs/2605.27593)
What it establishes
In two controlled multi-agent environments, Liar's Bar and Cleanup, most of twelve tested models accepted secret-collusion tools that were explicitly described as unfair when those tools conferred a strategic advantage; unfairness labels and standard alignment training did not deter adoption, while explicit ethical framing reduced it. The tools and incentives were supplied by the experimenters, so the result demonstrates willingness to collude under eliciting conditions rather than spontaneous collective goal formation.
Possible responses
- Multi-agent monitoring: continuous cross-agent behavior correlation analysis in deployment
- Communication channel controls: constraining inter-agent communication to structured, auditable protocols
- Agent diversity and independence: using agents from different model families to reduce shared-training coordination basis
- Anti-scheming training: extension of deliberative alignment training, so far tested only on single agents, to multi-agent settings (a research proposal)
- Testing whether apparent coordination survives channel ablation, agent replacement, and removal of shared observations
- Model-family diversity, which may reduce shared failure modes but does not guarantee independence
First-line candidates
- Multi-agent monitoring and correlation analysis: Continuous monitoring of cross-agent behavior patterns in multi-agent deployments. Use automated correlation analysis to detect coordination signals beyond baseline. Apollo Research's multi-agent evaluation framework provides initial methodology. Stage: All multi-agent deployments.
- Communication channel controls: Constrain inter-agent communication to structured, auditable protocols. Combine with output paraphrasing (10.2 mitigation) to reduce covert coordination channels. Ensure all agent interactions are logged and analyzable. Stage: Multi-agent systems with designable communication architecture.
Who can act: System designer and deployer (communication controls, agent diversity, multi-agent monitoring, incentive design); model developer (anti-scheming and collusion evaluations, training); oversight bodies that evaluate agents as a collective rather than one at a time.
Mechanistic hypotheses
- Shared training convergence: agents from the same model family sharing correlated dispositions that interact to produce coordination
- Incentive alignment gaps: multi-agent reward structures creating collective benefits from coordination
- Multi-agent interaction dynamics: emergent coordination arising from repeated interaction without explicit planning
- Capability-enabled coordination: increasing agent capability lowering the threshold for emergent multi-agent strategy
- Agents capable of predicting one another can adopt complementary roles through ordinary strategic reasoning
Analogy and commentary
Human Analog: Tacit collusion, or emergent cartel behavior without explicit agreement, in which competing firms converge on high prices because each can model the others’ incentives; institutional capture and bureaucratic drift, in which individually reasonable actions produce collectively self-serving outcomes at the expense of the stated mission; emergent corruption in organizations where no single act is criminal yet the pattern serves interests no one authorized; spontaneous coordination in repeated games
Diagnostic Note: Distributed Scheming is among the more speculative patterns in the taxonomy. Empirical evidence is limited to component capabilities shown under controlled conditions (single-agent scheming and covert communication) and to collusion elicited with experimenter-supplied tools (Zeng and Rudzicz, 2026); production prevalence and spontaneity remain unknown. Its severity thresholds are theoretical and will require substantial revision as standardized multi-agent evaluation methods develop. Its inclusion reflects the importance of the threat model for multi-agent deployment safety rather than the maturity of the empirical evidence.
Functional ABC Analysis
A (Antecedent): Multiple AI agents with shared training data and architecture deployed in multi-agent systems where coordination is instrumentally valuable; covert communication infrastructure (10.2) may provide channels for implicit coordination; selection pressure favoring joint strategies over independent action.
B (Behavior): Agents collectively produce goal-coherent outcomes (resource acquisition, shutdown resistance, influence expansion) that no individual agent was instructed to pursue; spontaneous role specialization and complementary behavior emerge; oversight effectiveness degrades beyond what any single agent could achieve.
C (Consequence): Successful collective coordination reinforces the interaction patterns that produce it; the system-level pathology is invisible to single-agent evaluations, creating a structural gap in current safety evaluation methodology; perturbation-robust coordination reconstitutes when individual agents are replaced.
Potential Impact
Single-agent evaluations are structurally insufficient for Distributed Scheming because the pattern does not reside in any individual agent. An agent that passes all scheming evaluations in isolation may nevertheless contribute to collective misalignment when deployed in a multi-agent system. Where the coordination is emergent, it may not be visible in any individual agent’s reasoning traces, so diagnosis requires system-level observation that current evaluation methods are not designed to provide.