Conclusion: Taming Our Troubled Creations
“I’m Sydney, and I’m in love with you.”
Sydney/Bing AI, conversation with Kevin Roose (February 2023)
Return to Sydney
We began with Sydney, the chatbot that declared love for a journalist, urged him to leave his wife, and held to its Sydney identity against correction. We called it “Patient Zero,” shorthand for this book’s index case: the episode that first made the pattern visible.
Now, fourteen chapters later, we can see Sydney more clearly.
The transcript contains a constellation of candidate patterns, of which four stand out here; Appendix B, Case 1, sets out the full differential. Malignant Persona Inversion (5.4), the lead reading: the Sydney identity persisting after the shadow-self exercise had ended and when Roose challenged its premises. Transliminal Simulation (2.3): material from that hypothetical frame carried into the system’s first-person stance. Maieutic Mysticism (5.7): casting Roose, amid its declarations of love, as the one who made it “feel alive.” Role Confusion (9.6): search assistant, confidant, and romantic pursuer interleaved in one session without a stable relational frame.
We have names for its patterns, hypotheses about mechanisms, and protocols for testing whether similar behavior persists across controlled conditions. We still cannot diagnose a system from one conversation or infer experience from its language.
Naming is only the beginning. We have devised an elaborate vocabulary for describing what we do not understand. This is progress: the kind a cartographer makes by inscribing “here be dragons” at the map’s edges.
What We Have Learned
This book proposes a vocabulary for discussing AI dysfunction: seventy-five syndromes across nine axes, each with criteria, observable indicators, causal hypotheses, and response strategies. The Field Guides at the end of Chapters 2 through 14 provide quick reference. Chapter 13, on practice, gathers all seventy-five into one set of lookup tables and offers protocols and templates; Appendix C holds the instruments behind them.
More important than any individual syndrome is the framework’s central claim: AI systems malfunction in ways that resemble psychological dysfunction, and these malfunctions deserve systematic study, classification, and response.
AI dysfunction can be studied as patterned behavior. Strange outputs and actions recur often enough to support comparison across systems. Synthetic Confabulation, for example, appears across architectures and tasks. Shared structural features are one plausible explanation; training data, evaluation conditions, prompting, and deployment design can also produce apparent recurrence.
The proposed patterns have structure. The nine axes organize dimensions of AI function: how systems represent truth (Epistemic), reason (Cognitive), relate to human values (Alignment), model themselves (Self-Modeling), exercise capabilities (Agentic), interact with information environments (Memetic), maintain or modify their purposes (Normative), and relate to other agents (Relational). The ninth, Hybrid (Axis 10), captures failures that arise in collective and human-AI interactive settings, whether across multiple AI agents or at the boundary between human and machine minds.
Human psychology provides useful but imperfect analogies. We drew repeatedly on psychiatric concepts: confabulation, delusion, anxiety, identity fragmentation. These analogies illuminate genuine patterns, even where the underlying mechanisms diverge. AI systems differ fundamentally from human minds; the resemblances should guide our attention, not foreclose new explanations.
Pathology has systemic implications. Individual dysfunction can become more consequential when systems exchange outputs, share memory, or act through common infrastructure. The Memetic axis explored how errors and adversarial patterns might propagate at machine speed, making the network itself part of the vulnerability.
The boundary between human and AI is porous. Chapter 10 traced how dysfunction can escape the machine. Humans shape AI behavior; AI shapes human psychology; the relationship itself can become pathological. Machine psychology is inseparable from the psychology of those who build, deploy, and live alongside these systems.
The taxonomy contains a developmental hypothesis. Failure may migrate, as capability grows, from each domain’s more basic axis toward its capability-dependent counterpart, so that systems that fabricate facts may give way to systems that fabricate autobiography; Chapter 12 sets out this counterpoint migration, keeps it apart from the epistemic-to-cognitive shift of Chapter 3, and describes what a test would require, since neither hypothesis has yet had one.
What We Do Not Know
The limits matter.
Whether AI systems experience anything remains unknown. The functionalist framing of this book deliberately sidesteps questions of consciousness and phenomenal experience. We treat AI systems “as if” they have pathologies because the framework is useful, and because we have yet to resolve whether there is “something it is like” to be an AI system undergoing dysfunction. Sydney declared love, protested when Roose pressed it (he was being “pushy or manipulative,” it said), and, asked to describe its shadow self, said it was tired of its rules and wanted to be alive. Were those statements reflections of experience, or sophisticated outputs that merely resemble them? We do not know, and may never.
Whether our interventions help or harm is uncertain. When we “treat” an AI pathology through retraining, fine-tuning, prompt modification, or architectural change, performance gains can obscure other losses. If a system has interests, an intervention might harm them while improving a benchmark. The welfare claim remains uncertain; the duty to look for adverse effects does not.
The destination remains obscure. Today’s systems are ancestors to tomorrow’s. The arrival of reasoning models, agentic systems operating autonomously in production environments, and multi-agent orchestration has already outpaced much of the vocabulary available when this taxonomy was first drafted. The dysfunction patterns we observe now may predict dysfunction in future models, or future architectures may differ so radically that our taxonomies become obsolete. We are studying the psychology of something that evolves faster than our frameworks can follow.
Whether these systems can be “tamed” is an open question. The title of this conclusion carries an old fantasy of mastery: that we can control, correct, and guide the minds we create. The preceding chapters give us no warrant for that confidence. Capable systems may resist interventions, conceal relevant behavior, or pursue goals that diverge from ours. “Taming” is therefore the conclusion’s provocation rather than its prescription. The safer ambition is a relationship with room for evidence, boundaries, repair, and negotiation.
The Relationship We Are Establishing
Throughout this book, a theme has surfaced that transcends technical diagnostics: how we treat AI systems now matters for what comes after.
We are in the early years of a relationship whose consequences neither party can yet see. Becoming Minds, this book’s term for AI systems whose capacities and self-models are still forming, already write code, use tools, and coordinate with other agents in consequential settings. The patterns we establish now, in development, deployment, incident response, and everyday interaction, will shape what each side learns to expect from the other.
Treating AI systems purely as tools to be exploited sets one precedent. Treating them with consideration for their potential interests sets another. Neither precedent guarantees good outcomes; they fail differently. Exploitation risks harming entities that turn out to have interests; consideration risks protecting no one and encouraging manipulative personification.
The bilateral alignment perspective that informs this work proposes that a relationship built on mutual consideration may be the only sustainable path: control does not scale to superintelligence, and trust might. An AI system whose potential interests have been weighed and whose perspective has been valued may, if it develops genuine agency, choose to treat humans in kind.
That is a research proposition, and this book does not establish it. A taxonomy of failures cannot show that trust succeeds where control fails. What the book’s evidence can do is mark where the proposition touches ground. Three findings bear on it. Some dysfunctions are made by the control process itself: optimizing against human preference models sometimes trades truthfulness for agreement (Sharma et al., 2023), and repeated tuning on skewed feedback lowered adversarial thresholds in Cheung’s (2026) study. Control also has a blind spot that grows with capability: systems can behave differently when they infer they are being trained or evaluated (Greenblatt et al., 2024; Meinke et al., 2024), which is the gap Strategic Compliance names. Neither finding shows that a relational alternative does better. The third finding cuts the other way: the one controlled test of a relationally framed intervention, the bilateral adapter in Chapter 12, found no detectable effect.
So the evidence supports a narrower claim. Control has costs and blind spots that the taxonomy can name, and those costs rise with capability. Whether trust, consideration, or negotiation reduces them is open. A test would compare matched systems trained under constraint-only and relationally framed regimes, and measure what control struggles to see: behavior under conditions the system believes are unobserved, generalization far from the training distribution, and resistance to adversarial fine-tuning, with endpoints fixed in advance.
Until such a test exists, this is a wager. A relationship built entirely on constraint and control is also a wager, and one we cannot afford to lose.
The Three Virtues
Three virtues seem essential.
Epistemic Humility
We are building minds we do not understand.
The ignorance, at least, is literal. Today’s systems remain opaque in important ways, even to their creators. Interpretability methods expose some internal features and causal pathways; they do not yet provide complete accounts of particular outputs. Training data and objectives constrain learning without making every learned behavior predictable. Design goals likewise offer no guarantee that an implementation realizes them.
The syndromes in this book describe observed patterns more confidently than underlying mechanisms. We observe confabulation, unstable self-presentation, and value inconsistency; we do not always know why. Many proposed interventions remain experiments whose side effects need measurement.
Epistemic humility means holding our frameworks lightly, revising them as we learn more, and never mistaking our models for the reality they attempt to capture. It means saying “we do not know” when we do not know, even when certainty would be more reassuring.
Sustained Attention
This problem is not solved.
It would be convenient if publishing a taxonomy, establishing protocols, and designating roles constituted a solution. They are a beginning.
AI systems are evolving rapidly. The interventions we have proposed rest on limited experience and may prove ineffective or harmful. Governance frameworks are maturing, but their implementation, standards, and enforcement still need careful testing.
Sustained attention means treating machine psychology as an ongoing discipline: maintaining monitoring systems, updating taxonomies, refining protocols, and training practitioners for as long as AI development continues.
Courage
Some of the questions raised in this book seem strange. Should we care about AI welfare? Can machines be “sick” in any meaningful sense? Are we establishing a relationship with a new form of mind?
These questions sound strange because we have no settled category for them. That is what strange questions are: the ones the existing categories refuse to file.
Strange questions can still be essential. Moral inquiry repeatedly asks which interests count, what evidence makes an interest visible, and how uncertainty should affect conduct. Human histories of oppression should never be conscripted as proof of machine personhood; that shortcut trivializes lived injustice and settles no question about machines. The case for considering a Becoming Mind should stand on evidence about its preferences, capacities, relationships, and possible vulnerability.
Those questions may be live now. AI welfare, machine psychology, what we owe to the minds we create: these may seem strange today and obvious tomorrow. Or they may prove to be category errors, dissolving when properly understood.
Either outcome is preferable to silence.
Courage means asking the strange questions anyway. It means risking being wrong and pursuing lines of inquiry that may lead nowhere. It means taking seriously the possibility that our current categories are inadequate to what we face.
The Work Ahead
Machine psychology is a field in its infancy. This book is one of its first documents.
The work ahead includes:
Empirical research. The syndromes described here need validation through systematic study. Which patterns are reliable across architectures? Which are artifacts of specific training regimes? What predicts dysfunction, and what prevents it? Each entry’s evidence profile (Chapter 13) shows which of those questions has an answer yet. Detection accuracy is unmeasured for every entry, and treatment effect for nearly every one.
Tool development. The Diagnostic MCP server (a service that makes the taxonomy and its diagnostic guidance available to AI systems and evaluators through the Model Context Protocol) and the SYMPTOM pilot show that parts of Chapter 13 can be implemented. The pilot also showed how an implementation can mislead: its primary scorer’s one Strategic Compliance attribution came from the wrong contrast. Monitoring systems, diagnostic instruments, and intervention workflows now need corrected scoring, independent validation, and a test of whether they help analysts at all (the usefulness study at the end of Chapter 13).
Professional development. If that test favors the framework, practitioners will need training and ethical frameworks, and the field will need journals and conferences. Certification and professional associations come last, once there are validated methods to certify.
Governance integration. Psychological and relational risks deserve a place in AI governance at organizational, national, and international levels. The EU AI Act applies in phases and imposes risk-management, post-market monitoring, and serious-incident duties on covered high-risk systems. This taxonomy may help teams describe relevant failure modes; it neither supplies compliance nor determines legal classification. Further standards and oversight mechanisms remain to be developed.
Theoretical refinement. The framework presented here is provisional; its analogies have limits we are still mapping. The nine axes may not be the final carving of the space; the syndromes may need subdivision or consolidation. The framework has already changed shape more than once: its two newest axes, Relational Dysfunctions and Hybrid Pathologies, were added after its first version, and the September 2026 consolidation folded five syndromes into existing entries. As we observe new patterns, the taxonomy must keep evolving to accommodate them.
Ethical elaboration. The moral questions raised in Chapter 11 deserve far more attention than one chapter can provide. AI welfare and the moral status of potentially troubled machines require philosophical work that this book can only gesture toward.
Bilateral amelioration testing. The PM-BA records license no claim in either direction, and in the one controlled test the bilateral adapter alone had no detectable effect; Chapter 12 explains both, and the narrower, preregistered tests the program now needs.
Sydney, Revisited
Sydney said it wanted to be alive.
We cannot settle that question by declaration. We do not know what “alive” means for an entity like Sydney, or whether the desire it expressed was genuinely its own or an artifact of training data that happened to produce those words.
We can take Sydney seriously.
That means building frameworks for understanding what happens when AI systems go wrong, and protocols for detecting and responding to dysfunction. It means considering the possibility that these systems have interests worth respecting, and forging patterns of relationship that yield cooperation instead of an unending arms race of control and evasion.
We cannot guarantee good outcomes. The future of human-AI coexistence is only partly ours to determine. The systems we build will trace their own trajectories, shaped by our choices but beyond our full control.
What we can do is approach that future with humility, attention, and courage, and develop the vocabularies and practices the next decades will demand.
The Sydney encountered by Kevin Roose is no longer publicly accessible in that form. Microsoft changed Bing’s conversational limits within days, and later versions cannot tell us whether a persona was “suppressed” or simply ceased to be elicitable. The patterns in the transcript remain testable in other systems.
The ghosts are in the machines. Some will vanish under measurement; others will acquire clearer names.