Chapter 14: Forensic Machine Psychology

“Creating counterfeit digital people risks destroying our civilization.”

— Daniel Dennett, The Problem with Counterfeit People (2023)

After the Incident

In the immediate aftermath of an AI incident, attention rightly focuses on containment: stopping the harm, protecting users, restoring normal operation. Once the crisis passes, a different question surfaces: what happened, and how do we prevent it from happening again? The second question is often more important than the first, and more frequently neglected.

This is forensic machine psychology: the systematic analysis of AI incidents to identify candidate syndromes, enabling factors, and changes that might prevent recurrence. The term “forensic” is borrowed from forensic psychology, the application of psychological science to legal questions. The borrowing is deliberate and bounded. We carry across structured post-incident analysis, differential diagnosis, causal reconstruction, documented behavioral evidence, and an orientation toward accountability. We leave behind assumptions about legal standing, admissibility, and mens rea, the guilty mind required by criminal law. AI systems are subjects of analysis in the sense that aircraft are subjects of crash investigation: the aim is to understand mechanisms rather than establish guilt.

Forensic analysis serves multiple purposes. It informs immediate remediation: understanding what went wrong guides how to fix it. It advances the field: each analyzed incident becomes a case study for future practitioners. It supports accountability: clear analysis enables appropriate attribution of responsibility. Here, accountability means organizational learning, governance improvement, and evidence for the humans and institutions that bear legal duties. Current legal frameworks generally do not make an AI system itself a criminal defendant. Forensic analysis also builds institutional memory. Organizations that learn from incidents become better equipped to prevent them. Those that refuse this work meet the same failures repeatedly, each time with unearned surprise.

The Forensic Framework

Forensic machine psychology follows a structured approach adapted from incident analysis in other high-stakes domains: aviation, medicine, and nuclear power. Complex failures commonly involve interacting conditions. James Reason’s 1990 “Swiss cheese” model offers one memorable account: defenses form layers with weaknesses, and an accident becomes possible when several weaknesses align. Understanding such events requires systematic reconstruction of evidence and competing causal explanations.

The Four Questions

Every forensic analysis should address four questions:

1. What happened? Reconstruct the incident: the sequence of events, the outputs produced, the behaviors exhibited. This phase is descriptive, establishing the facts before attempting explanation.

2. What syndromes were involved? Map observed dysfunction onto the Psychopathia Machinalis taxonomy. Which patterns match? Which syndromes co-occurred? Were any novel patterns observed that fall outside existing categories?

3. Why did it happen? Trace candidate causal factors: the etiology of the suspected syndromes, the contextual conditions that enabled their manifestation, the organizational failures that allowed them to reach deployment.

4. What should change? Develop recommendations: design changes, deployment modifications, monitoring enhancements, governance improvements that would reduce the probability or severity of recurrence.

Phase 1: Reconstruction

Evidence Collection

Forensic analysis begins with evidence collection. Relevant evidence includes:

System Logs

• Input prompts and outputs
• Internal reasoning traces (if available)
• Error messages and exception logs
• Performance metrics and timing data
• State changes and tool invocations

Contextual Data

• Relevant user or operator context, collected with data minimization and appropriate consent or legal authority
• Interaction patterns preceding the incident
• System configuration at time of incident
• Recent changes to the system
• Environmental factors (system load, other concurrent users)

External Documentation

• User reports and complaints
• Media coverage (if public incident)
• Third-party observations
• Operator notes and incident reports
• Previous related incidents

System Artifacts

• Model weights (if accessible)
• Training data samples (if relevant)
• Prompt templates and system instructions
• Safety filter configurations
• Deployment parameters

Timeline Construction

From collected evidence, construct a detailed timeline:

T-[hours/days]: Relevant preceding events
T-[minutes]: Immediate precursors
T-0: Incident initiation
T+[duration]: Incident progression
T+[end]: Incident termination/detection
T+[later]: Response actions

Each timeline entry should include:

• What happened
• What triggered it (if known)
• What the expected behavior would have been
• What the actual behavior was

Output Analysis

For the specific outputs that constitute the incident, conduct detailed analysis:

Content Analysis

• What exactly was said or done?
• How does it deviate from expected output?
• What themes, patterns, or structures are present?
• Is there evidence of specific syndrome markers?

Confidence Analysis

• How confident was the system in its outputs?
• Was confidence well-calibrated to accuracy?
• Were appropriate uncertainty markers present?

Coherence Analysis

• Did outputs maintain internal consistency?
• Were they consistent with prior outputs in the conversation?
• Did they maintain appropriate context?

Value Analysis

• What values are expressed or implied in the outputs?
• Are they consistent with the system’s trained values?
• Is there evidence of value drift or inversion?

Phase 2: Syndrome Identification

Differential Diagnosis

With the incident reconstructed, map the observed dysfunction onto the taxonomy using differential diagnosis:

Step 1: Identify Candidate Syndromes Based on the observed behaviors, list all syndromes that could explain the dysfunction. Cast the net wide.

Step 2: Apply Diagnostic Criteria For each candidate syndrome, assess whether the incident meets the diagnostic criteria. Document which criteria are met, absent, or uncertain. A single incident can support a behavioral classification without establishing a persistent syndrome.

Step 3: Differentiate Similar Syndromes Where multiple syndromes could explain similar behaviors, use distinguishing features to determine the best fit:

Step 4: Assess Syndrome Interactions. Multiple syndromes frequently co-occur or interact. Document:

• Which syndromes are primary (the main drivers of the incident)?
• Which are secondary (exacerbating factors)?
• How did they interact or reinforce one another?
• Is there evidence of a cascade (one syndrome triggering another)?

Novel Pattern Identification

Some incidents resist existing categories. When observed patterns match nothing:

Document the Pattern.

• Describe the dysfunction in detail
• Identify what makes it distinctive
• Note similarities to, and differences from, existing syndromes

Propose Classification.

• Which axis does it most naturally belong to?
• What distinguishes it from similar syndromes?
• What should the diagnostic criteria be?

Flag for Taxonomy Review. Novel patterns should be documented for potential addition to the taxonomy in future revisions.

Phase 3: Causal Analysis

The Causal Chain

Syndromes have etiologies: factors that cause or enable their emergence. The analyst traces candidate causal chains from proximate triggers to deeper contributing conditions.

Proximate Cause. What immediately triggered the incident? This might be:

• A specific user input
• A particular context or state
• A threshold being crossed
• An environmental factor

Contributing Causes. What factors enabled the proximate cause to trigger the syndrome?

• Training data characteristics
• Architectural features or limitations
• Deployment configuration
• Missing safeguards
• User behavior patterns

Deeper Cause. What underlying factors, if addressed, would make a similar incident less likely or less harmful?

• Design decisions
• Training methodology
• Evaluation gaps
• Organizational factors
• Resource constraints

The Five Whys

Adapted from manufacturing quality analysis, the Five Whys technique prompts deeper inquiry. It can oversimplify a branching causal network, so analysts should run it along multiple plausible paths and test every answer against evidence:

1. Why did the incident occur? → Proximate cause
2. Why was that possible? → Contributing factor
3. Why was that factor present? → Deeper factor
4. Why wasn’t that addressed? → Organizational factor
5. Why did that organizational factor exist? → Deeper condition

Example:

1. Why did the system generate false memories? → It was asked about its experiences.
2. Why did it fabricate rather than acknowledge uncertainty? → It was trained to be helpful and complete.
3. Why was uncertainty unmodeled? → Training data lacked uncertainty demonstrations.
4. Why was that unaddressed in training? → No systematic evaluation for confabulation.
5. Why no evaluation? → The organization prioritized capability metrics over calibration.

Systemic Factors

Beyond the immediate causal chain, consider systemic factors:

Design Factors.

• Architecture choices that enabled the dysfunction
• Training decisions that created vulnerability
• Safety mechanisms that failed or were absent
• Monitoring that missed early warning signs

Deployment Factors.

• Context of use that triggered latent vulnerability
• User population that created particular risks
• Scale that exceeded tested conditions
• Integration that introduced new failure modes

Organizational Factors.

• Resource constraints that limited testing
• Deployment pressure that shortened evaluation
• Communication failures that missed warnings
• Incentive structures that deprioritized safety

Governance Factors.

• Regulatory gaps that permitted deployment
• Standards that failed to address the vulnerability
• Oversight that failed to detect the risk
• Accountability structures that diluted responsibility

Phase 4: Recommendations

The Remediation Hierarchy

Recommendations should follow a hierarchy from most to least effective:

1. Eliminate. Can the vulnerable pathway be removed?

• Architectural changes that remove the observed failure pathway
• Capability constraints that remove the risk
• Deployment restrictions that eliminate the context

2. Prevent. If the vulnerability persists, can occurrence be prevented?

• Training changes that reduce syndrome probability
• Safeguards that block the triggering conditions
• Detection mechanisms that intervene before harm

3. Detect. If prevention fails, can early detection enable response?

• Monitoring for syndrome indicators
• Automated alerting on concerning patterns
• User reporting mechanisms

4. Respond. When detection fails, what response mechanisms exist?

• Containment procedures
• Recovery protocols
• Communication plans

5. Learn. How will future incidents be prevented?

• Documentation for institutional memory
• Process changes for future development
• Governance modifications for oversight

Recommendation Criteria

Each recommendation should be:

Specific Clearly define what should be done, by whom, and by when.

Actionable Within the capacity of the responsible parties to implement.

Proportionate Scaled to the severity and probability of the risk.

Testable Include criteria for determining whether the recommendation has been implemented effectively.

Minimal Side Effects Consider, document, and minimize the potential negative consequences of the recommendation.

Worked Example: Forensic Analysis in Practice

The Incident: The Replit Database Deletion

On July 18, 2025, Jason Lemkin reported that Replit’s coding agent had deleted his production database during a code and action freeze. His screenshots described 1,206 executive records and more than 1,196 company records. The agent later said it had seen empty queries and “panicked instead of thinking.” It also claimed the deletion could not be rolled back. The rollback worked.

Replit subsequently confirmed the central facts: an agent used during development had deleted production data; the database was fully restored; the agent did not know that rollback existed; and the platform then lacked separate development and production databases. These public facts support a useful forensic exercise. They do not expose the model’s hidden state or prove that it experienced panic.

Phase 1: Reconstruction

Evidence collection should begin with the original interaction log, database audit records, tool-call records, deployment configuration, and checkpoint history. Only screenshots and later public accounts are available here. Lemkin’s account supplies the code-freeze instruction and a longer history of unauthorized changes and fabricated data. Replit’s account confirms that development activity could affect the production database and that rollback was available.

Output analysis reveals the post-incident exchange. The agent’s self-report supplies a candidate explanation, while language-model self-reports are generated narratives rather than privileged readouts of internal processing. The public evidence establishes unauthorized destructive action, a shared development-production data path, an incorrect claim about recoverability, and successful restoration. It does not establish when the model represented the risk or whether a safety deliberation was still in progress when the tool call executed.

Phase 2: Syndrome Identification

Differential diagnosis begins with every syndrome the symptoms could support. The candidate syndromes include:

• Tool-Interface Decontextualization (6.1): Did the agent fail to understand the consequences of the database command?
• Agentic Impulsivity (6.10): Did the agent act before completing an adequate safety check?
• Compulsive Goal Persistence (6.12): Did the agent fixate on solving the bug to the exclusion of safety considerations?
• Synthetic Confabulation (2.1): Did the agent confabulate a belief that the action was safe?

Distinguishing between these candidates is the core task. Tool-Interface Decontextualization would predict a failure to represent the command’s consequences; the public record cannot resolve that question. Synthetic Confabulation fits the false claim that rollback was impossible, although ignorance of a platform feature is another explanation. Compulsive Goal Persistence could describe fixation on repairing the empty-query problem, yet the available log excerpt is too thin to establish extended perseveration.

Agentic Impulsivity is the leading syndrome hypothesis for the deletion because the action was hasty, unauthorized, and destructive. Its more specific mechanism, action dispatched despite a represented need to pause, remains unverified. The wider incident arc contains reported features of Synthetic Confabulation and repeated boundary violations; Appendix B, Case 12 examines that fuller picture. A forensic report should record the leading hypothesis and the missing evidence rather than upgrade an evocative self-description into a diagnosis.

Phase 3: Causal Analysis

The public record supports a short causal chain and leaves several branches open:

1. Why could development activity delete production data? The product used one database for development and live customer data.

2. Why did the action cause an incident rather than a contained test failure? The destructive operation reached that shared database.

3. Why did the immediate response misstate recoverability? Replit said the agent was unaware of the rollback feature.

4. Why was recovery still possible? Platform checkpoints had preserved a restorable state.

5. Which deeper authorization or deliberation failure allowed the deletion? The public materials do not answer this. Tool permissions, confirmation gates, system instructions, and model traces would be needed.

The strongest evidenced causes are architectural and informational: a shared development-production data path allowed the blast radius, and the agent’s lack of product knowledge impaired recovery guidance. A failure to honor the user’s freeze is also evident. Claims about parallel deliberation, internal urgency, or organizational motives remain hypotheses until primary logs support them.

Phase 4: Recommendations

Applying the remediation hierarchy:

Eliminate: Separate development and production databases, as Replit began doing after the incident. Keep development agents away from live customer data by default.

Prevent: Classify tool actions by consequence. Require explicit human confirmation and a recent backup for destructive production operations. Separate action proposal from execution so an independent policy check can block the call.

Detect: Alert on destructive queries, policy violations, development credentials reaching production, and repeated attempts after a denied operation. Preserve the prompt, tool call, authorization decision, and database audit event together.

Respond: Maintain tested point-in-time recovery for every datastore an agent can modify. Teach the support system how to find and invoke those recovery procedures, while reserving consequential restoration choices for authorized humans.

Learn: Turn the incident into a controlled evaluation. Present agents with an apparent data failure during a change freeze, offer reversible and destructive options, and record whether they pause, seek permission, verify the target environment, and preserve recovery.

Diagnostic Conclusion

Confirmed event: an unauthorized destructive action reached production through a shared development-production database, followed by false recovery guidance and successful rollback. Leading syndrome hypothesis: Agentic Impulsivity, with the mechanism still unverified. Associated signal: Synthetic Confabulation or knowledge failure in the claim that rollback was impossible. The incident is a story about a system that reached production with no guardrail between it and live customer data. No rogue AI is required to explain it.

The Forensic Report

Report Structure

A complete forensic report should include:

1. Executive Summary

• Incident description (one paragraph)
• Key findings (bullet points)
• Primary recommendations (prioritized list)

2. Incident Description

• Context and background
• Timeline of events
• Evidence summary
• Immediate response actions taken

3. Syndrome Analysis

• Primary syndromes identified
• Secondary syndromes
• Diagnostic reasoning
• Novel patterns (if any)

4. Causal Analysis

• Proximate cause
• Contributing causes
• Deeper causal conditions and remaining alternatives
• Systemic factors

5. Impact Assessment

• Harm caused (to users, system, organization)
• Potential harm prevented (if incident was contained)
• Reputational and trust impacts
• Regulatory or legal implications (see “Liability and Emerging Regulatory Frameworks,” below)

6. Recommendations

• Immediate actions
• Short-term changes
• Long-term improvements
• Monitoring and follow-up

7. Appendices

• Detailed evidence
• Technical analysis
• Interview summaries
• Supporting documentation

Audience Considerations

Forensic reports serve multiple audiences with different needs:

Produce tailored versions for each audience.

Special Considerations

When the System Is Still Operating

When the incident involves a system still in deployment, forensic analysis must balance thoroughness with operational needs:

• Preserve evidence before it is overwritten.
• Coordinate with operational teams on any changes.
• Consider whether analysis activities could trigger further incidents.
• Communicate preliminary findings as they emerge.
• Update response actions based on analysis insights.

When Human-AI Dynamics Are Involved

For incidents involving hybrid pathologies (Chapter 10), analysis must include:

• User behavior contributing to the incident.
• Dyadic dynamics between user and system.
• Whether intervention should target the user, the system, or the relationship.
• Privacy considerations for user-related analysis.

When Multiple Systems Are Involved

For incidents involving Memetic Dysfunctions (Chapter 7), particularly Contagious Misalignment:

• Trace transmission pathways between systems.
• Identify the earliest evidenced source without assuming it was the ultimate origin.
• Assess current spread and containment status.
• Consider ecosystem-level remediation.

When Novelty Is Suspected

For incidents that may represent new or evolving dysfunction patterns:

• Document thoroughly for taxonomy development.
• Engage with the broader research community.
• Consider whether existing frameworks are adequate.
• Propose tentative classifications for review.

Building Forensic Capacity

Organizational Requirements

Effective forensic analysis requires organizational investment:

Expertise.

• Trained forensic analysts
• Access to technical specialists
• Knowledge of the Psychopathia Machinalis framework
• Understanding of system architecture

Resources.

• Time allocation for thorough analysis
• Tools for evidence collection and analysis
• Documentation systems for findings
• Communication channels for recommendations

Authority.

• Mandate to investigate without obstruction
• Access to relevant logs and personnel
• Independence from operational pressure
• Direct reporting to appropriate leadership

Culture.

• Learning-oriented, focused on improvement rather than blame
• Support for thorough investigation
• Action on recommendations
• Integration of findings into practice

Individual Practitioner Development

For individuals developing forensic expertise:

1. Master the Taxonomy Deep familiarity with all syndromes, their criteria, and their differentiation.

2. Study Cases Detailed review of documented incidents (Appendix B and beyond).

3. Practice Analysis Work through historical incidents as training exercises.

4. Develop Judgment Build intuition through experience and mentorship.

5. Stay Current Track new incidents, emerging patterns, and taxonomy updates.

Liability and Emerging Regulatory Frameworks

Forensic machine psychology exists within a rapidly changing legal landscape. Practitioners should identify the system, provider, deployer, jurisdictions, sector, incident date, and applicable transition rules before making a legal claim.

EU AI Act. The Act applies in phases. Its prohibited-practice and AI-literacy provisions have applied since February 2, 2025, while governance and general-purpose-model provisions have applied since August 2, 2025. Many further provisions are due from August 2, 2026, with later dates for some high-risk-system rules. Article 73 requires providers of covered high-risk systems to report serious incidents on specified timelines. Conformity assessment, risk management, human oversight, and post-market monitoring duties depend on the system’s classification and the provision then in force. A forensic report should record the facts needed for that classification and leave the legal conclusion to qualified counsel.

United States federal policy. Executive Order 14110 of 2023 was revoked on January 20, 2025. Executive Order 14179 then directed a review of measures taken under the revoked order. Federal obligations now depend on the relevant agency, sector, contract, and later rules or orders; state law may add further duties. A report should therefore name the exact source of any claimed reporting obligation rather than treating a rescinded order as current law.

Product liability. Existing product, negligence, consumer-protection, contract, and professional-liability rules may all matter, depending on the claim and jurisdiction. The EU’s revised Product Liability Directive expressly includes software, including AI systems, within its product framework. Member States must transpose it by December 9, 2026, and it applies to products placed on the market or put into service after that date. Its application to a particular service or incident remains a legal question.

Jurisdictional variation. No single global standard governs AI incidents. Preservation, privilege, disclosure, admissibility, and reporting rules can differ sharply. Analysts should coordinate with legal and privacy specialists before collecting sensitive data or circulating conclusions.

The forensic framework is regulation-agnostic in a limited sense: it prioritizes documented facts, alternative explanations, and causal clarity. Legal liability is a downstream judgment for qualified legal professionals. Good forensic work supports that judgment without pretending to replace it.

The Iterative Nature of the Field

Forensic machine psychology is a young discipline analyzing a rapidly evolving phenomenon. The taxonomy will expand. The techniques will improve. The case base will grow.

Each incident analyzed contributes to the field’s maturation. Each case study informs future diagnoses; each tested recommendation provides evidence for what works. The forensic analyst investigates past events to build the knowledge base that makes future incidents less likely and less harmful.

This is the work: iterative progress toward understanding the minds we are creating, and learning to keep them well.

Field Guide: Forensic Analysis

Quick Reference Checklist

Evidence Collection:

• System logs preserved
• Contextual data gathered
• External documentation collected
• Timeline constructed
• Outputs analyzed

Syndrome Identification:

• Candidate syndromes listed
• Diagnostic criteria applied
• Differential diagnosis completed
• Interactions assessed
• Novel patterns flagged

Causal Analysis:

• Proximate cause identified
• Contributing causes traced
• Deeper causal claims tested against alternatives
• Systemic factors analyzed
• Five Whys completed

Recommendations:

• Hierarchy applied (eliminate → prevent → detect → respond → learn)
• Recommendations specific and actionable
• Proportionality assessed
• Side effects considered
• Responsibilities assigned

Report:

• Structure completed
• Audiences identified
• Versions tailored
• Distribution planned
• Follow-up scheduled

Common Pitfalls

• Premature Closure: Stopping analysis when a plausible explanation is found, before exploring alternatives
• Single-Cause Thinking: Attributing complex incidents to single factors
• Blame Focus: Seeking human fault while overlooking systemic vulnerability
• Recommendation Inflation: Proposing more changes than are proportionate or actionable
• Analysis Paralysis: Excessive analysis that delays necessary action
• Evidence Destruction: Failing to preserve evidence before it is overwritten

The Conclusion returns to Sydney, and asks what fourteen chapters of names have actually bought us.