If a TSA agent deports someone simply because the machine recommended it, then that is a problem. But if a TSA agent deports someone simply because they were having a bad day, then that is also a problem.
I guess I don't care what tools those in authority are using (their intuition or a mechanical intuition or a database lookup), but what I care about is whether or not innocent people without the ability to navigate an appeal system are being erroneously penalized.
And ultimately, it's the responsibility of management to make sure that the people/tools they assign are doing a good job. If they fail, then management needs to find new people/tools. If management fails to do that, then all the AI regulation in the world isn't going to do much good.
However, if they hire a bunch of power hungry sociopaths who are very good at hiding their malicious oppression and who also bring in donuts every Thursday to stay on their bosses good side, then the situation could easily lead to worse outcomes for the people who have to deal with this system.
If we create a computer system that oppresses 1% of innocent people, then that is a problem. However, I don't consider it a win to ban the computer system and replace it with a human system that oppresses 10% of innocent people. Like, the situation isn't better because humans are oppressing humans instead of a computer doing the oppression.
That's why I was focused on management. I don't care that things are going badly for some specific technology related reason. It's management's job to fix it regardless. If management can't rely on the technology for regulatory reasons, then they might rely on people who do just as bad of a job. And hey, that scenario is even better for management because if they hire a bad actor who gets caught then that person faces the consequences and not them.
You're saying "I don't trust AI, I want human supervision", but this works both ways. Sometimes we don't trust the humans and would prefer a neutral AI. Humans do terrible things to other humans. Who's going to review my appeals? What are their biases? Are they any more trustworthy than a model?
Take the example of predicting terrorism threats based on facial queues of stress or fear. Machines lack context, which a qualified human would otherwise take into consideration. You can be stressed out because you might be accompanying a child, or fearful that you might miss a flight. If a TSA agent deports someone simply because a machine recommended it, that would be inhumane.
People like to argue that more regulation will adversely affect automation and/or growth/scaling of technologies and businesses. Growth is important but it must not cost us our humanity.