datawithaplot.org · clinical vs. actuarial prediction
You are running pretrial intake this morning. Ten people were arrested, released, and given a court date. Your job: for each file, predict whether they return to court or miss it.
Read however you like. The history, the job, the notes, the vibe. Use everything.
No takebacks. Once you call a file, it's called. When you finish, a checklist that uses exactly two facts makes the same ten calls, and we compare scores.
All ten files are fictional composites. Nothing you enter leaves your browser.
Make the call, then the next file opens.
This is everything it looked at. It never read the notes.
This fight is old. In 1954, the psychologist Paul Meehl compared expert predictions with simple statistical rules across every study he could find, and the rules kept winning. Almost fifty years later, a meta-analysis of 136 studies reached the same verdict: mechanical prediction ties or beats expert judgment most of the time, in medicine, education, and courtrooms alike.
The reason is not that experts know too little. It is that files like these are full of vivid, useless detail. The apology letter, the anger, the exhaustion: they feel like signal and behave like noise. Two boring facts carry most of the real information, and the checklist never gets distracted from them, never has a rough morning, never gives one extra chance because someone brought their mom.
There is also a quieter trap in the stack: most people return to court. Guess "returns" every time and you score of 10 here without opening a single file. A gut that calls a lot of misses has to be very good just to beat doing nothing.
The lesson is not that judgment is worthless. Someone had to pick the two variables, and someone should keep studying the exceptions the rule gets wrong. The lesson is where judgment belongs: building the checklist and auditing it, not overriding it one charming file at a time. When intuition and a validated rule disagree, the rule is the better bet, and it does not get tired by the tenth file.