At what? At correctly diagnosing patients or predicting prognosis once an issue has already been identified, as in Meehl's paper? That is not the same thing as determining whether a patient has the "mental capacity" to commit suicide.
The common objection against this evidence is "sure, it happened to work there, but it won't work in this case!" And of course, it's very hard to defend this research against an accusation like that.
However, it's been a remarkable stable result whenever it is tried for a new task, so I guess what I'm saying is that I like to stay open to simple algorithms making better decisions than humans. Not because the algorithms are good (linear models with random weights are not very good) but because humans are incredibly fallible.
But going beyond that, Kahneman in his latest book summarises the recent research in the area as such:
- Simple linear models generally outperform experts.
- Simple linear models generally outperform experts when the experts get additional information to base their decisions on, that the model does not get.
- Simple linear models generally outperform experts when the experts get to know and use the outcome of the linear model.
- Simple linear models trained only on an expert's judgments and not the actual outcome outperform the very expert they were trained on.
- Simple linear models with random weights (!) outperform experts.
- Simple linear models with equal weights (i.e. transform the predictors to the same scale and then just sum them) outperform experts.
- Simple linear models with equal weights and almost all predictors removed except the best 1--3 outperform experts.