This is Gizmodo's business model. It is a glorified blog. They don't do original reporting. Gizmodo was originally a property of Gawker, and thus the model is essentially commentary and opinion on the reporting generated by other organizations.
Because Gizmodo is really just an opinion/editorial blog, it doesn't really attempt to provide unbiased or fact driven reporting. So it is then no surprise that the actual facts align quite poorly with the headline.
And it looks like they had some data:
>We examined 23,631 predictions generated by Geolitica between Feb. 25 to Dec. 18, 2018 for the Plainfield Police Department (PD). Each prediction we analyzed from the company’s algorithm indicated that one type of crime was likely to occur in a location not patrolled by Plainfield PD. In the end, the success rate was less than half a percent. Fewer than 100 of the predictions lined up with a crime in the predicted category, that was also later reported to police.
That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. So it's still what I would expect even if it was reasonably good.
I wish the article would have provided more details about why the PDs chose not to use it. Was it because it was bad at predictions? Cumbersome to use? Glitchy?
It reminds me of a project I was involved with that used a "real-time" computational fluid dynamics model to optimize datacenter air-conditioning. Management was big on the hype, but if you paid attention, the system was routinely unplugged from the actual control system because the facility engineers just found it too difficult to work with.
Why on Earth would we want people who have a poor grasp of the law, paired with the means and the license to arrest and kill be transparent about their work..?
E.g., I’d like to live in a works where lawyers are concerned with justice rather than “winning” but I’m not shocked when they act like winning is the goal.
How morality informs the law and the ways lawyers should behave (i.e, should they phone it in/decline the case in if they disagree ethically) is interesting but more difficult as a topic
With all that said, I think the original point still stands: in most people's minds, an 'ideal' system is one where all the lawyers are trying to seek justice. We just create a less-than-ideal adversarial system as a weak proxy for that given that human nature tends to make the ideal an unreliable expectation. I don't think that's fundamentally different from the PD transparency issue.
If a defense lawyer knows that his client is guilty, and successfully defends him anyways, that means there was not sufficient evidence to convict. Don't blame the defense lawyer for doing his job rigorously. Blame the prosecution for failing at theirs.
This assumes the jury is rational and objective. There's plenty of behavioral psychology that goes against this, but a more glaring observation is that lawyers do not seek to maximize the objectivity of the jury during selection. It's usually very much the opposite.
The goal of the district/trial court system is finding of fact. That's objective. Winning is about swaying the jury, which pulls on the levers of subjectivity. The system is, in part, designed to protect the rights of the innocent. Take the Blackstone ratio, which assumes it is better to let 10 guilty people go than convict 1 innocent. Verdicts cannot be purely objective in a system designed to be asymmetrical.
That’s partially my point. Ideally, yes, humans should act impartially and irrespective of incentive. That’s not the world we live in, though.
(1) Is the software significantly predictive? Test the software's predictions against actual crime in areas with no patrols.
(2) Are patrols effective deterrents to crime? Observe the area in question with and without patrols. (I wouldn't be surprised to learn there are already such studies.)
If both (1) and (2) yield positive results, you can then use the software to direct patrols and see if the method itself is effective at reducing crime; which would also serve to further confirm (2).
Of course, even if only (1) yields positive results, there would probably be other benefits to using the software to direct patrols, like reducing response times (which could also be tested).
The study of those questions and the reliability of your answers to those questions is called statistics and it is an entire branch of mathematics, usually with its own department at most universities.
The wonderful Weapons of Math Destruction has a chapter on how this leads to a self reinforcing loop.
More crimes means more police presence. More police presence means more recorded crimes. More crime data means more police …
This does nothing to negate the value of data used intelligently.
Yes you can use data well. It is unfortunately not being used well.
The article in the headline implies the opposite: statistical inference of future crime location -> more cops in said location -> no crime in said location.
It should be fairly easy to back out an out of sample data set for this - ie predicted crime location where the extra cops didn't get deployed and see if that lines up with empirical observations.
(The right number is probably extremely small, because crime is very bad {{citation needed}} and even a small chance of prevention is useful.)
Thankfully, police are notoriously fair-minded and empathetic, and never arbitrary, deceptive, or biased when it comes to probable cause.
Uh, seems a lot like it should be the government answering that question before spending tax money on it and subjecting real people to the uncaring US justice system just because crime is bad so we have to do something, even if that something is ML snake oil?
When one says that machine learning software is terrible, you have to trust either the officers, or their boss. I have a feeling that the same people who criticize the software trust rank-and-file officers even less..
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html
(the parent was originally posted to https://news.ycombinator.com/item?id=37753079)
> A new joint investigation by The Markup and Wired...
And when I go to the page about actual investigation by The Markup [1]
> Our investigation stopped short of analyzing precisely how effective Geolitica’s software was at predicting crimes because only 2 out of 38 police departments provided data on when officers patrolled the predicted areas. Geolitica claims that sending officers to a prediction location would dissuade crimes through police presence alone. It would be impossible to accurately determine how effective the program is without knowing which predictions officers responded to and which ones they did not respond to.
Also, later in the article
> Plainfield officials said they never used the system to direct patrols.
Given all this, it's somewhat simplistic to say it's "pretty terrible at predicting crimes", even though that makes for a good clickbait headline. It seems that the software was intended to identify high-crime areas that to target for patrolling, which doesn't seem like a huge problem to me -- but it seems like the software was never actually used as intended in the first place.
----------------------------------------
[1] https://themarkup.org/prediction-bias/2023/10/02/predictive-...