This is very much my worry with AI in government. Those systems mass hallucinating fraud or some other punishable crime, and then the justice system doing what it does by threatening absurd sentences to compel plea deals and the evidence takes years to be properly challenged.

We are implementing AI in places it shouldn't be used and this whole thing is going to spin out of control at some point.