Sure I don't disagree with this. I guess the point is not so much that ML models are bad here in the typical sense (although greedy consultants may, and IMO likely are, happy to pawn off shitty models to jurisdictions which don't know any better), but more that the underlying system isn't one where predictive modeling is truly going to be "effective" (although, as another commenter pointed out, there are cases where predictive modeling works fairly well such as pretrial detention risk assessment). The problem as I see it is that model "efficacy" means one thing to the cops and voters ("effective" in the sense of reducing or preventing crime) and another thing to a data science ("effective" in the sense of able to achieve a high F1 score or w/e). These definitions may be correlated, but are not guaranteed to be, and the strength of the correlation is highly dependent on how the model is ultimately used.
Re:Predictive Policing Software Is Pretty Terrible at Predic...
by tech_ken on Tuesday October 03, 2023 @06:15PM
(#37758348)
> The problem is the externalities of the policy. Not some mode overfitting.
But it would be very easy to do something like to predict the probability that a cop makes an arrest given they went to each area. And if there’s no difference in the areas, it should not matter how many times they went there. The rates should be the same.
It seems likely that the models were right and that it’s way easier to make drug arrests in these areas, which was kind of baked into the original premise. So it’s not clear why blaming the modeling is an issue here.
The problem is the externalities of the policy. Not some mode overfitting. Like would you blink if I told you the probability of being able to make a drug arrest in a poor area was 20% higher? Probably not. Does that need to mean that you only go to the poor area? No.