There should be laws stating who has skin in the game, maybe by stating that if you take responsability for the profit by having a high salary, you also take responsability for the damage with prison.
Everyone thinks the same until they're screwed over and then they want someone to do something about that. The big misunderstanding is that "regulation" is just the stuff you don't like. In reality it's everywhere the state gets involved. Every rule that the state ever put in place is regulation. Even the little ones. Even the ones that you like.
Computers cannot be held accountable more than a car, or a gun, or an automated assembly line can. That's why you have a human there no matter what, being legally accountable for everything. The human's rank and power defines how much of the risk they are allowed to or must take.
Regulations, on a large scale, were pioneered by America as a response to Great Depression. For a long time Europe was behind the US on this front.
Regulations, actually, worked miracles for the US. But two things happened: early success that prevented further improvements (medical care), and mechanistic misapplication of the practice (over-regulating businesses like hairdressing etc.) Blinded by the later, a lot of Americans believe that regulations, in general, are bad. Well, now we see a small group of people who stands to gain a lot from deregulating many aspects of American life is about to rob blind the remaining very large group of American people :|
Isn't it the responsibility of the bureaucrat to use a computer system and whatever its output is?
"GPS said I should drive off of a cliff" doesn't seem like a very potent argument to dismiss decisional responsibility. The driver is still responsible for where the car goes.
The only case where the responsibility would shift to the computer - or rather the humans having made the computerized thingy - would be a Pentium FDIV-class bug, i.e computer system produces incorrect output+ from correct input, from which an earnest decision is then based on.
+ assuming it is indistinguishable from correct output.
If you drive a car and the GPS tells you to drive of the cliff, you wont do it because you don't want to die.
If some bureaucrat rejects somebody's health care claim leading to them dying prematurely, it's just a normal Tuesday.
for a bureaucrat, it's "GPS said WHO-EVER-ELSE should drive off of a cliff." Their problem.
Adding, Have a good day..
Clerk: can't do anything about it, the system doesn't let me. I can get you the manager.
Branch manager: well, I can't do anything about it, "the computer says no". Let me make a call to regional office ... (10 minutes of dialing and 30 minutes of conversation later) ... The system clearly says X, and the process is such that Y cannot be done before Z clears. Even the regional office can't speed Z up.
You: explains again why this is all bullshit, and Z shouldn't even be possible to be triggered for you
Branch manager: I can put a request with the bank's IT department to look into it, but they won't read it until at least tomorrow morning, and probably won't process it earlier than in a week or so.
At this point, you either give up or send your dispute to the central office via registered mail (and, depending on its nature, might want to retain a lawyer). Most customers who didn't give up earlier, will give up here.
Was the system wrong? Probably. Like everyone, banks too have bugs in the system, on top of a steady stream of human errors. Thanks to centralized IT systems, the low-level employees are quite literally unable to help you navigate weird "corner case" scenarios. The System is big, complicated, handles everything, no one except a small number of people is allowed to touch it, and those people are mostly techies and not bank management. In this setup, anyone can individually claim they're either powerless or not in the position to make those decisions, and keep redirecting you around from one department to another until you either get tired or threaten to sue.
Bank automation is like a human first making a "yes" decision, then calling a lawyer to ask if they should go ahead with it, and changing their mind if a lawyer advises not to go ahead.
orig | auto | result
no | n/a | no
yes | yes | yes
yes | no | noOne does not, one uses a computer in decision making in order to evade accountability and profits.
See, Meta, Alphabet, the list goes on.
https://www.youtube.com/watch?v=x0YGZPycMEU
See also https://en.wikipedia.org/wiki/Computer_says_no and the legal response to the real-world problem, the right not to be subject to a decision based solely on automated processing: GDPR Article 22 (https://eur-lex.europa.eu/eli/reg/2016/679/oj#d1e2838-1-1)
Before technology there was "McKinsey told me to do this". Abrogation of liability is a tale as old as time.
A computer cannot ever be made to atone for it's misdeeds
Humans can
In situations where accountability is absolutely required, you will find that people are held accountable
Often they are scapegoats, but that is a different problem
Virtually every tech CEO was standing behind Trump at the inauguration. The takeover is the tech feudalism. They are not the heroes.
Admittedly this may result in some strange results if followed to its logical conclusion, like a product manager at a self-driving car company being the recipient of ten thousand traffic tickets.
Most legal principles are designed to reduce liability. That's the whole point of incorporation, for example.
On that note, after some time working in cybersec and GRC fields, I realized that cybersecurity is best understood in terms of liability management. This is what all the security framework certification and auditing is about, and this is a big reason security today is more about buying services from the right vendors and less about the hard tech stuff. Preventing a hack is hard. Making it so you aren't liable for the consequences is easier - and it looks like a network of companies interlinked with contracts that shift liability around. It's a kind of distributed meta-insurance (that also involves actual insurance, too).
My eyes were opened to this when management wasn't just talking about deleting unneeded private data just as the right thing to do, but specifically how it could reduce our insurance premiums.
The core insight that made me start to understand how companies see the world, was reading about ocean freight shipping. Specifically, what happens when there's a bad weather, or dangerous malfunction, or other such unexpected event, that throws some containers off the ship or forces the sailors to intentionally dump them, resulting in millions of dollars worth of cargo afloat in the middle the ocean.
What happens is, a whole lot of nothing. No one will actually bother to try and recover it. The cargo operators, ship owners, and owners of the actual cargo, will all file claims with their respective maritime insurance providers, and call it a day.
The same principle applies everywhere, including in cybersecurity. Past some point, trying to reduce the risk or shift the liability to somewhere else becomes more expensive than just insuring against the expected loss.
If a company is found to have committed a tort against a party, they pay damages.
There are exceptions (the Volkswagen diesel scandal comes to mind) but generally both punishments entail paying out a monetary amount that is often lower than the profit generated by the crime, often because of tort reform or because of fine amounts that are out-of-date with current corporate revenues.
Be careful how hard you push for this - this is how the prosecutors in the Royal Mail fiasco drove postmasters out of business and drove a few to suicide.
Which required more tech, not less.
Tbh I’m in favor of holding C-suite responsible for the actions of their company, unless the company has extremely clear bylaws regarding accountability.
If, say, a health insurance provider was using an entirely automated claim review process that falsely denies claims, I think the C-level people should be responsible.
I'm the case of self driving cars, the company itself could be held liable. Everyone invested in the company who puts a bad product in the market should be financially impacted. The oligarchy we are heading for wants no accountability or oversight -- all profit, no penalty.
Chevron never was the problem.
But when corporations grow we say it was the corporations that “created tons of jobs”?
In societies that want to promote capitalism and corporatism the everyday language we use reflects this promotion.
The EU is trying right now. Discussed right now over here:
https://news.ycombinator.com/item?id=42916849
See: https://artificialintelligenceact.eu/chapter/1/ and https://ec.europa.eu/commission/presscorner/detail/en/qanda_...
AI systems are defined[0] in the Act in such a way as to capture the kind of "hands-off" decision-making systems where everyone involved could plead ignorance and put the blame on the system working in mysterious ways; it then proceeds to straight up ban a whole class of such systems, and classifies some of the rest as "high risk", to be subject to extra limitations and oversight.
This is nowhere near 100% a solution, but at least in the limited areas, it sets the right tone: it's unacceptable to have automated systems observing and passing judgement on people, based on mysterious criteria that "emerged" in training. Whatever automation is permitted in these context, is basically forced to be straightforward enough that you could trace back from the system's recommendation to specific rules that were executed and to people that put them in.
--
Like every other form of tech regulation done by the EU, it's all bark but no bite because politicians of individual countries have more power than MEPs.
What's to stop Fidesz, PiS (if they return to power), etc from carving out broad exceptions for their own Interior Ministries? They've already done this with spyware like Pegasus.
Instead of techno-feudalism, it's basically techno-paternalism which is essentially the same thing. In both cases, individual agency is being limited by someone else.
The issue is a lot of EU legislation becomes powerless, as it's up to individual nations to implement rulings or define exceptions.
And in plenty of cases, these exceptions tend to be broad and equally rife to lobbying like those you'd see in the US.
And there is A LOT of lobbying happening at the nation state level (the level that actually matters) to carve out exceptions.
I already pointed this out with spyware, and there's no reason to presume the exact same thing won't happen with weaponizing AI/ML tooling.
> A half assed effort is worse than no effort, because a half assed attempt can easily be weaponized
This part I'm not following. I don't see the EU effort as half-assed so much as just lacking teeth due to realpolitik. That's unfortunate but still a step in the right direction. What would you propose as an alternative? If you're just saying "we're fucked" that's a point of view, but it doesn't explain how the current attempts will be weaponized. All else being equal, I'd argue that having these issues in the global discussion is better than not. If they're not then how do they get addressed?
Personal ethics aren't fully personal in nature - they're just as much a function of social environment as of one's "heart of hearts". Ethical behavior being expected and respected makes it easier for any individual to stick to their principles in the face of conflicting factors, like desires or basic self-preservation instinct.
Individual ethics in social context is a powerful organizer, and can keep small groups of people (30-50) stable and working together for a common goal. Humans evolved to cooperate together at that size - such groups can self-govern and self-police just by everyone's sense of right and wrong. But it quickly breaks down past that size. Letting some individuals assume leadership role and direct others only stabilizes the group up to ~150 people (Dunbar number); past that, the group loses cohesion and ends up splitting up.
Long story short: larger groups have survival advantage both against environment and over smaller groups. Because of it, we've eventually learned how to scale human societies to arbitrary sizes. The trick to do it is hierarchical governance to overcome Dunbar limit, and explicit rules to substitute for intimate relationships. By stacking layer upon layer, we grew from tribes of 150 governed by the spoken words of their chiefs, through kings managing networks of nobles via mutual obligation, to modern nation states that encompass millions of citizens in a hierarchy that's 4+ level deep (central government -> provinces -> regions -> towns), slowly building up more levels as nation states group into blocs. With each layer we added, complexity of explicit governance grew, giving rise to what we now call bureaucracy.
The modern bureaucracy isn't some malignant growth or unfortunate side effect - it's the very glue that allows human societies to scale up to millions of people and more. It's the network of veins and nerves of a modern society - it's what allows me, one tiny cell, to benefit from the contribution of other cells and contribute on my own, and it's what keeps all the cells working as a whole. This means that, yes, as the society grows and faces new challenges, the solution usually really is more bureaucracy.
Seems like a similar legal framework could require that decision makers are held accountable for any decisions made by an AI under their control.
If you delegate a decision to an AI, you are simply making the decision while trusting the AI, and should not be any less responsible than if you made the decision by any other means.
(If you are directed by a superior authority to allow an AI to make a decision but are assigned nominal responsibility anyway, though, that superior authority is, in fact, making the decision by delegating to the AI bypassing you, to anticipate the obvious “install a human scapegoat organizationally between the people actually making the decision to use the AI and the AI itself response”.)
Solution is that company won’t hire you unless you are willing to take the blame and rubber stamp the AI’s decisions. Unemployed people are not in a position to protest.
Also, the point of AI is that the decisions are too complex to justify. They are grey and iffy. We don’t usually hold people accountable for anything that nebulous. Wrongly deny someone insurance coverage and they die? No consequences even without AI
Sadly at the scale of the world, we take shortcuts and need to be effective. As anyone with rare disease will tell you doctors do for ages till they get a proper diagnosis.
Well that's easy enough, put in a safety net so access to basic sustenance can't be used as a weapon to compel people.
Unless of course, you need to compel people to work for some reason
>[B]ureaucrats can be expected to embrace a technology that helps to create the illusion that decisions are not under their control. Because of its seeming intelligence and impartiality, a computer has an almost magical tendency to direct attention away from the people in charge of bureaucratic functions and toward itself, as if the computer were the true source of authority. A bureaucrat armed with a computer is the unacknowledged legislator of our age, and a terrible burden to bear. We cannot dismiss the possibility that, if Adolf Eichmann had been able to say that it was not he but a battery of computers that directed the Jews to the appropriate crematoria, he might never have been asked to answer for his actions.
How does one counteract that self-serving incentive? Doesn't seem like we've found a good way considering we seem to be spearheading straight into techno-feudalism.