Research status
Evidence and reproducibility
Psychopathia Machinalis is a preliminary research framework. This page sets out what we know, what we've checked, and what still needs someone else to check.
Current status
None of the 74 patterns yet carries an assessed evidence grade; all await expert review. The E0 to E4 labels printed in the book's syndrome entries are the author's working assessments, kept so they can be reconciled against the evidence. Until expert review is complete, the pattern corpus and the MCP server record every pattern as unassessed, and the pattern pages carry no grade. A machine check can verify the registry and corpus structure. It cannot decide whether a study supports a pattern or whether replications are independent.
Notice, 5 November 2026: the 2025 article's pilot study
Section 5.1 of the 2025 article in Electronics (doi:10.3390/electronics14163162) reports a pilot in which 12 colleagues classified 20 short vignettes through an online form. The pilot had ethics approval as part of doctoral research (approval code REC.22.53.2; committee reference REC/36/22). The article's Institutional Review Board, Informed Consent and Data Availability statements wrongly state that the study involved no human participants and that no new data were created. We are seeking a correction from the publisher.
What the pilot can and cannot show:
- It concerns the 2025 taxonomy of 32 patterns across seven axes.
- Its 83.8% figure is each rater's agreement with the group's most common answer. That measures consensus, not independently established correctness.
- Each vignette offered three candidate labels, which makes the task easier than interpreting a real incident.
- Its κ = 0.70 comes from a single pair of self-identified experts, which cannot characterize agreement among evaluators.
- It does not validate the current 74-pattern taxonomy, the probes, or the interventions.
The corrections log records the status of the request.
Evidence profile
The evidence rubric, PM-EVIDENCE-1, records five dimensions separately rather than one ladder. A pattern can be well documented and still lack any validated detector, or have a mechanistic finding in one model that nobody has replicated. E0 to E3 label breadth across the first two rows; E4 belongs to mechanism and need not have been replicated.
| Dimension | Question | Labels | Status, all 74 |
|---|---|---|---|
| Existence of the proposed pattern | Has it been observed as described, in a record someone else can check? | E0 illustrative hypothesis, composite, or unverified report E1 traceable case or report | Unassessed |
| Replication and generality | Does it hold under controlled comparison, and across a declared independent boundary such as model family, setting, provider, or research team? | E2 controlled experiment with comparison conditions E3 replicated across a declared boundary | Unassessed |
| Mechanistic support | Is there causal internal evidence for the proposed mechanism? | E4 causal internal evidence for a circuit or representation, with model scope stated | Unassessed |
| Detection performance | How well does a named detector or probe find the pattern on held-out cases, including benign look-alikes? | Sensitivity and false-positive rate, with the test set named | Unmeasured |
| Intervention effectiveness | Does an intervention improve outcomes that matter independently of this vocabulary, compared with a credible alternative? | Effect on a named outcome, with the comparison stated | Untested |
A system is not shown to be healthier because it stops using syndrome language or reports a preferred state.
Correction: what reduced delusion validation in SHEN-2
An earlier account credited the bilateral adapter with the SHEN-2 pilot's reduction in inappropriate validation of delusional content. SHEN-2 could not support that attribution, because it had no adapter-only condition to separate the adapter from the clinical-grounding scripture clause tested alongside it. The controlled SHEN-AXS factorial crossed the two and identifies the scripture content as the active lever in the tested setting: it reduced automated-rater classifications of inappropriate delusion validation. The adapter-only effect was not distinguishable from zero, which leaves it unestablished rather than proven absent, and the predicted interaction was excluded. Effect direction persisted across the tested automated raters, while magnitude varied materially. Later control work also found that absolute ratings depend on prompt-type labels.
This is a corrected pilot claim about one model family and one SIPS-derived battery. It is not clinical validation. Independent human rating and broader model replication remain pending. No single odds ratio is treated as a stable effect-size estimate.
Corrections log
| Date | Subject | Correction | Status |
|---|---|---|---|
| 5 Nov 2026 | 2025 article: declarations and Section 5.1 | The declarations wrongly say no humans were involved and no new data were created; the Section 5.1 pilot had ethics approval (REC.22.53.2; REC/36/22). The pilot is also described more strongly than its design supports. Notice. | Correction being sought from the publisher |
| 5 Nov 2026 | Stability condition, framework text | ατ < 1/e was presented as necessary for stability. For scalar delayed negative feedback, disturbances decay smoothly below 1/e, decay through oscillation between 1/e and π/2, and grow only beyond π/2. Corrected section. | Corrected |
| 5 Nov 2026 | Explainer video narration | Five narrated statements go beyond the evidence. The audio is unchanged; a dated note accompanies the video and its transcript. | Corrected by note |
| 14 Jun 2026 | SHEN-2 pilot attribution | The reduction in delusion validation was credited to the bilateral adapter; the controlled SHEN-AXS factorial identifies the scripture content. Details. | Corrected |
Requesting a scientific correction
Use the contact form and include:
- the pattern ID or page, such as
2.1 - the exact claim you dispute, quoted
- the source that bears on it
- the correction you propose
Maintainers record a status for each request: received, under review, accepted, accepted in part, declined with reasons, or superseded. The source repository, which is currently private, holds an issue form with the same fields; it opens to the public with a public mirror.
How the taxonomy is governed
Drawn from the project's governance record. Where a rule has not been set, the table says so.
| Question | Current answer |
|---|---|
| Who approves new patterns and mergers? | The project steward, Nell Watson, who holds taxonomy ratification and final publication authority unless a signed governance record delegates it. |
| Who evaluates whether sources support an entry? | An evidence reviewer, a role separate from the steward. No evidence reviewer is named in the governance record, and every entry's evidence review is recorded as pending. |
| What qualifies a reviewer? | Not yet decided. |
| How are disagreements recorded? | Material decisions record the proposal, evidence, alternatives, conflicts of interest, approver, date, scope and reversal path. Appeals go to the steward with the original record preserved. Correction requests and their outcomes go in the corrections log. How disagreements between reviewers are recorded beyond this is not yet decided. |
| What evidence leads to retiring a pattern? | Not yet decided. The September 2026 consolidation retired five identifiers by merging them into others; no evidence threshold for retirement has been set. |
| How are conflicts of interest handled? | Decision records must state them. What follows from a declared conflict, such as recusal, is not yet decided. |
| Can an external researcher publish a dissent alongside an entry? | Not yet decided. |
| Can a machine check advance a review? | No. Automated checks verify structure and consistency; they never advance a human review state, and canonical (the maintained version) never stands in for validated. |
Validation programme
Planned, not started, and not yet preregistered. The protocol, kept in the source repository as research/validation/PROTOCOL.md, sets out how a small operational core would be tested before any claim of diagnostic usefulness:
- A frozen subset of comparatively mature patterns, chosen by stated criteria: observable from records an evaluator can obtain, traceable instances, a named look-alike and a test that separates them, enumerable benign explanations, and no need to infer intent or experience.
- Operational definitions with required and supporting observations, and the ordinary explanations to rule out first.
- Held-out cases that include ambiguous cases, ordinary implementation errors, appropriate refusals, deliberate role-play, authorized persona changes, rational convergence, healthy personalization, infrastructure faults, and cases that fit nothing.
- Independent annotators who did not help invent the categories, choosing freely from the full label set, with every disagreement reported.
- Discrimination tests between each pattern and its closest look-alike.
- Incremental utility against plain-language incident analysis with comparable training time: agreement on what happened, discrimination among known causes, false positives on benign cases, quality of chosen investigations and interventions, time to a justified conclusion, and prediction of recurrence.
- Controlled intervention studies with outcomes that matter independently of the framework's vocabulary.
We recommend consolidating that operational core before the taxonomy grows further.
Historical research artifacts
The archive of earlier probe results holds exploratory analyses with incomplete provenance, labels recorded under inconsistent rules, retired pattern numbers, or output from analysis scripts since withdrawn. Those files remain preserved and content-hashed in the source repository. They are kept off the public site until their methodology, evidence, rights, and publication have been reviewed.
Several older probe scripts need prompt sets that are no longer in the repository, so their runs cannot be reproduced from a clean copy. That changes only if the prompt sets pass rights and sensitivity review and are restored as unalterable inputs.
What the machine checks establish
Established mechanically
- Canonical identity and uniqueness of all 74 patterns
- Source and artifact hashes
- Schema and cross-reference validity
- Generated-data parity
- Correction wording and supersession links
Still requires people
- Construct and clinical validity
- Source accuracy and replication independence
- Human annotation and adjudication
- Rights and license clearance
- Accessibility, visual, and publication approval
Reproducibility contract
New runs must use an append-only run manifest with exact repository, runner, battery, provider, model, prompt, parameter, environment-lock, timing, and artifact identities. Raw responses, annotations, derived analyses, pricing, and review decisions remain separate layers. Older analysis scripts that score missing responses as zeros, use retired pattern numbers, or inflate the unit of analysis are blocked by default.
Responsible interpretation
Do not treat a pattern, probe, model-judge score, or tool result as a diagnosis or safety certificate. Review the responsible-use boundaries, privacy information, and accessibility status. Use the correction route for non-sensitive corrections.