"Caffeine: do stupid things faster and with more energy!"
Likewise, these new tools enable us to make entire new categories of mistakes based on false correlations etc., and to make them more often. Before, it would have taken a lot of work to cancel thousands of people's health insurance. Just the sheer amount of time and number of eyeballs involved would have increased the chance that someone might have noticed a problem. Now it's just >click< and there's no chance for introspection before the results of bad data or bad analysis are made manifest in the real world. Some gears are meant to grind slowly.
(1) Grease the wheels significantly (allowing these decisions kinds of humiliations to occur much more quickly, and in greater bulk);
(2) Enable "action at a distance", both physically and psychologically (it's a lot easier for anyone at any level in the bureaucratic apparatus to shrug their soldiers and say "well, the computer flagged him, I just gotta do my job" than to make a personal determination about that person's guilt or criminal association;
(3) And they add a patina of (false) respectability, via the extreme sexiness of machine learning and everything data-sciencey these days.
None of the stories show that. For example, the health one could have happened just as easily in a paper bureaucracy - that's the magic of 'opt in' vs 'opt out'. Just declare invalid all health insurances which are not explicitly renewed. No computer necessary.
> (2) Enable "action at a distance",
What, like paperwork doesn't enable that?
> (3) And they add a patina of (false) respectability
The flagged problems have more to do with no staff and the bureaucracies being self-serving in refusing to review complaints and whatnot, they don't have to do with regular people going 'well, the machine said it, so it must be true!'