Run it multiple times with multiple defendants, set up a control group that's receiving remote advice from actual lawyers, mask which group is which to the judges, then ask the judge(s) at the end to rank the cases and see which defendants did best.
That would be a lot more work, but it would also be much higher quality data than what they were trying to do.
And also
> That would be a lot more work, but it would also be much higher quality data
I don’t know much about the field of law, but anecdotally it doesn’t strike me as particularly data driven. So I think, even before introducing any kind of AI, the above would be met with a healthy dose of gatekeeping.
Like the whole sport of referencing prior rulings, based on opinions at a point in time doesn’t seem much different than anecdotes to me.
But I’d love to be proven wrong though.
That said. The consequence of most traffic tickets is increased insurance and a fine. Yes these do have an impact on the accused, but they are the least impactful legal cases, so it would make sense to focus on them as test cases.