If someone does do this, I'd be very interested in reading the outcome. Sounds like a fantastic idea. There's a few people who I've seen post on HN before that are familiar with filing FOIA requests. If the dataset can be acquired and released, I'm pretty sure there's enough people on HN who might be willing to work together in the open to write the analysis.

Admittedly, I didn't read the article. That said, I'm very familiar w/ the FOIA process and have a legal team setup to send them out; if the volume isn't huge then the cost isn't tremendous.

If you could provide more specific information to me on what would be beneficial/useful to FOIA, I'd be more than happy to take a look, and likely send them -- and put them up on a public Github repo with a license that makes the data useful.

Email me -- arthur [at] dantonio [dot] co [no M].

I would be willing to donate (a small amount) for these expenses. If it comes together, please put me in any mailing lists or message me directly.

Ditto.

ditto

Count me in; if I can help I will. I live outside the US but I suspect this problem probably exists in countries all around the world.

Is there a ranking of countries by fairness of their justice system, done by a neutral party? Would be interesting to know where countries stand.

http://worldjusticeproject.org/rule-of-law-index ?

Count me in as well. I'm freshly unemployed and looking for fun projects, preferably with a side of fighting for great good!

jonathan [dot] j [dot] mason [at] gmail

That's a really interesting suggestion.

Straight off the top, one problem is that transcripts aren't routinely generated, AFAIU. Yes, there's a court reporter taking down the proceedings, but what's generated isn't itself directly intelligible. Preparing an actual transcript from the proceedings for a small (2-day) case can run close to a thousand dollars. Doing this at scale would likely be highly expensive.

You bring to mind an alternative though: an automatic, random, appeal of a select number of cases to a review system of some sort. This might initially be a completely random sampling across all courts, but could then be subject to some sort of stratified sampling, with more cases subject to review from areas highlighted as problematic, and periodic reweighting.

What form does the "not directly intelligible" transcript take ? If it's possible to obtain that maybe there could be a way to generate an intelligible transcript automatically.

It has to be processed and formatted. The reporter uses a shorthand system that needs to dealt with, and you need to be able to figure out which party is which. Also, I'm pretty sure the reporter just blasts on through, so it would be difficult to distinguish between cases without some human intervention. You probably have hundreds of people getting arraigned in one session in the Bronx.

Digital audio can be more difficult. Automated transcription sucks, and the costs for human annoyed machine transcription are very high.

I don't know. I do know that preparing a relatively short transcript (in terms of court proceedings) can be a nontrivial request -- e.g., it's not simply pressing "generate" on some system.

That said, the records I've seen have been accurate.

Could potentially use cases for which both proceeding notes _and_ transcripts are available to train a sentiment analysis system to interpret the proceeding notes directly.

A further complication -- my understanding, this may be incorrect: court reporters are in at least some cases independent contractors. They hold rights to their transcripts. You're not actually dealing with the court in requesting them.

Your AI solution would 1) tremendously reduce consts and 2) almost certainly remove a large share of the income of these independent reporters.

Mind: the problem could be resolved by making reporters employees of the court.

Don't forget about a population density. I suspect a lot of the 'ratial' bias in the justice system stems from the number of interactions with poor people in high vs low population areas. In some areas you could sit on your back porch target practicing with a shotgun and nobody would care. That's not the case in any city.

Get drunk and drive off the road into a corn field, well if you can get out without anyone noticing then no big deal.

Several years ago(before it was against ToS) I scraped data from a county clerk website and did some mild analysis on it. I won't say what I was looking for, but I did find several correlations between case disposition (charges dropped, reduced, etc....) and certain combinations of judges/prosecutors/defense attorneys.

One particular combination showed a 10x increase in correlation with charges being significantly reduced or dropped.

It would be very interesting to see a proper dataset and analysis on this type of information.