If you mean the mental/emotional state necessary for a human to act without regard for consequences of their actions to other people, then computers are not ruthless.
If you just mean acting without consideration of how their their actions will affect people then computers fit this definition of ruthless perfectly by virtue of not being able to consider the consequences of their actions on other people, because we don’t know how to program them to consider how their actions will affect other people.
We don't have to. For example, a ruthless train door could be made less ruthless, by having sensors that respond to someone who is nanoseconds late, to open the door, just that once, just like it's elevator door brethren, but only once.
The problem is that "inherently opportunistic people" will take advantage of the machine kindness and take an entire train full of people hostage.
The oppressive train door is a dictator everyone loves, maybe if they were also razor sharp, people would give them the respect they deserve. phwump. guillotrains. on time, every time.
I don't understand, what am I missing?
I've never seen a computer have any regard for the consequences of its decisions. Computers are thus completely ruthless.
edit: typo
There's also unexplored (by article) concept of inadvertent or emergent control.
E.g. Dept A encodes Rule 1 and Dept B independently encodes Rule 2, yet applied together Rule 1 and 2 produce an unexpected outcome. In which case, neither Dept A or Dept B (the ostensible controllers) could be said to actually control the outcome.
I think the oppressiveness will come from software. Who writes it, and what values does the software reflect? Does it reflect yours?
The governance structure wants to code their rules into a machine that will execute those rules without being able to apply human discernment. Humans are meant to be out of the loop - their rules and diktats are meant to be all we need. Ai + robotics has brought this possibility onto the horizon.
The issue really then, is in people's acceptance that someone can write meaningful rules for them to follow. Hence we see jobs such as AI ethicist. As if they can do anything other than justify the immoral decisions corporations/government make.
The collective that still believe government has their best interests at heart and are unable to refuse following the worst of us, will drive us into technological tyranny.
The point of the article boils down to something like this:
Pre-AI computers are very literal and show no judgement or discernment in their rule-following. They are inevitably used to implement complex systems against which users have little recourse, and to implement complex systems whose behaviours and failures can be blamed on computers rather than on the people responsible. These fundamental problems will persist no matter the progress made in AI.
Free Software is intended to stop the user being oppressed by their own computers. The article's point is about other people's computers being used to serve their interests over yours, in ways that may be detrimental to the user compared to a more 'manual' system.
Consider the train door example from the article. Suppose that code were Free Software. This might benefit whoever owns and runs the train, but wouldn't benefit the user (of the train) at all.
Partly, but this example was also used as well:
> Though perhaps unlikely, my hyper-awareness of the sinister ruthlessness of machines in the ordinary things around us led me immediately to note the pathological outcome: Someone stuck on a desert island (or equivalent) who starves to death, because the only food they have is expired Juicero juice bags and a machine which refuses to serve them. While in Juicero's case this might seem like a comical example, I fully believe it's only a matter of time until something like this happens. The deliberate utilisation of machine-assisted ruthlessness to malevolent ends becomes more and more common with every passing day, as companies realise they can escape from accountability by hiding behind the facelessness of the machine, which disempowers the individual to fight or object to it. The now memetic expression, “computer says no”, pithily expresses this phenomenon.
…which is precisely the usual free software issue. The Juicero is a computer you own, but that does not serve your interests.
Don't make me tap the sign: Technology. Is. Not. Neutral.
A shame so few technophiles have read Jacques Ellul[1]. Ellul's La Technique (and his later "Propaganda") should be mandatory reading for anyone thinking about systems. Be careful though, this thinker/philosopher has verbalized a problem that made Ted Kaczinsky conclude the only solution is for us as a species to retreat into the forests and vanish into the trees. Read Ellul at your own risk.
About the rest I agree, but this is complaining about a system that works _exactly_ as intended. Read Ellul!
[1] "La Technique" (The technological society) https://archive.org/details/technologicalsoc00ellu
While Ellul separates technique from the individual machines that contribute to its reign, he maintains that machines represent an ideal state for the forces that govern technique. Humans, on the other hand, are not ideal at all. We lack the efficiency and predictability of machines. Therefore, Ellul says, technique is the social adaptation that exists in transforming the messy lives of humans so that they better fit into a world controlled by machinery. Further, in a somewhat terrifying turn of phrase, he adds that technique is "the consciousness of the mechanized world."
So humans adapt to technology and as technology becomes more efficient we just become cogs of technology is the core principle here and I can see how this relationship could be thought to arise however I completely disagree with it. Technology does enable efficiency at a fundamental level but it's not that humans become slaves to the efficiency or technique, it's that some humans chose to force other humans into increasingly more efficient means.
There is some feedback relation here in that in a global competitive environment as leaders around the world continue to compete in various forms they will inherently drive their populations into deeper efficiency I believe but the rise to all of this is because of the global competitive environment. It's largely social constructs and to some degree, finite resources that drive these behaviors and lead to efficiency focus. More than anything it's greed and hunger for power that drives this nonsense.
Most humans recognize when their lives become dominated by some silly efficiency and don't care to follow suit. Their want autonomy, to own their life and own their choices. They're instead forced into "the technique" so to speak as a secondary effect. The underlying effect is what needs to be rutted out, technology isn't the fundamental issue here, it's a set of human behaviors and institutions that exist and enable them.
Well, to build things at the modern computer industry scale, you need government bureaucracies, huge corporations, tons of discipline, from the assembly line to the cobalt mines, and enterprises and bureaucracies with a use for it, 99% to streamline control, control more effectively and wider, and penny pinch customers even better.
Now, give surveillance, instant indexed cataloguing, 24/7 tracking, remote controllable money and id access and more, to the same governments and corporations, and that's just going to point to more control.
There are six laws.
1) Technology is neither good nor bad; nor is it neutral.
2) Invention is the mother of necessity.
3) Technology comes in packages, big and small.
4) Although technology might be a prime element in many public issues, nontechnical factors take precedence in technology-policy decisions.
5) All history is relevant, but the history of technology is the most relevant.
6) Technology is a very human activity-and so is the history of technology.
You can't feed people with an AR-15.
The metaphors that define computing are all about abstraction, automation without human input, predictability, and control.
Abstraction tends to lossy and context-insensitive, which is why an algorithm will say "no" to a loan request based on a credit score that ignores personal context, where a human loan manager who knows the people involved might say yes - for good and valid reasons.
Computing seems to be one of the most successful products of modernism, which values speedy predictable repetition, lack of decoration, and anonymised mechanisation.
It's possible to imagine computing in other value systems. But it's revealingly hard to do without losing the essence.
All of which is a complicated way of agreeing - computing is not neutral, and very probably can't be.
It's the first automated value evangelising system in history. And that's not a comfortable thought.
A human loan manager, given only a credit score, will approve the loan based on only the credit score. The problem isn't human vs machine, it's context-awareness and context-sensitivity vs a lack of one or both.
All large systems, human or machine, tend to be implemented using abstractions. Layers of context-stripping abstractions were a part of buerocracy far before machines took it over. If a loan request form doesn't have an "if I don't get this loan my family will literally starve to death" box that you can check, the evaluator will not know about that context couldn't possibly act based on it. The form can be digital or paper and the evaluator can be a program or a human - it doesn't change a thing.
Similarly, context-sensitivity (the willingness to let the context sway your decision) was strictly limited even back in the day when decision makers were close enough to the source to be aware of the context. A bank clerk had to stick to the rules and couldn't just decide to give a loan to someone that didn't qualify under them. If they did, they'd be risking losing their job or worse, so they didn't. This discretion was only given to the people in charge - and that hasn't changed with computers. A person in a high enough position can override what the computer says, just as easily as they can override a lower clerk's decision.
Yes, making machines context-aware takes some effort, and making them context-sensitive takes even more effort. But even if machines magically came with both, their operators would do the best to get rod of them, just like they did with their employees. From their point of view, empathy is inefficiency. And that's the real problem.
"Gas chambers are just a tool"
But a tool that has very few uses besides large scale genocide. So of course it would be ridiculous to argue the gaschambers are just a tool, because they are (A) the result of an engineering project that tried to kill as many people as possible in a short time, without using valuable war resources like amunition, so the "tool" cannot be isolated from the context of it's creation, and (B) there is one thing gas chambers are good at and it is mechanizing genocide. One could potentially come up with other things they could be used for, but there is just the one thing theyr really excell at.
Computers have more uses, but they are "not just a tool" either, because in reality nothing is.
Yes, exactly.
Absolutely. I vehemently disagree with this essay. One of the reasons I am a passionate technology critic and sceptic is that I love technology and feel disgusted at the way it's co-opted and abused.
I am always vigilant not to become jaded or lose sight of the immense positive possibilities for digital technologies. But I sincerely believe that before that can happen and we can move into the true golden era of technological humanism, Big Tech, surveillance culture and over-bearing governments must be eliminated.
> it's inherent to the people and institutions making use of them.
I disagree. It resides within them and is amplified by technologies. "Inherent" implies that it is an indelible, fundamental feature. Not at all. Humans, as individuals and institutions can do better. We're just seeing amplification and facilitation of the same old human weaknesses, laziness, greed, stupidity, fear - go pick up some Aristotle, or any religious text for a comprehensive list of moral failings.
> Computers are neutral technology.
Not quite. All technologies carry values in their design. An AK47 and a baby incubator carry different values, one is a for taking life and one is for preserving it. At a more subtle granularity computers carry values with them. I recommend Edwin Black's "IBM and the Holocaust" for some insights here. A Google Pixel carries different values than a Raspberry Pi - though to most eyes the differences are invisible.
> It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are.
Yes, computers have enabled tyranny. That's hardly debatable now. I call it techno-fascism. Other people use different terms. It's there in everything from Facebook to "cashless commerce".
> The computers belong to them now
So long as computers are controlled by software, which is mutable, the more complete and devastating will be the reversal when we take them back. There is always poetry and irony in ultimate justice. One way of looking at it is the tyrants are simply building the apparatus to enable the greatest and most audacious bloodless revolution in the history of mankind. The future of freedom depends on "hackers".
Sell them more rope.
Plenty of great reading: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C44&q=alg...
Although I am not sure that captures the problem of technofascism.
Algorithmic/Cybernetic Governance is not malicious. It's just stupid. It's a mistake made by well meaning but misguided and poorly read people who have a vicariously omnipotent fantasy that machines can run things. Ultimately it's their intent to step back and bask in an autonomous land of milk, honey, magical plenitude and obedience.
Technofascism has a malicious and power hungry side. For that lot, computers are simple weapons of enslavement.
I also think the Cyberneticians are more honest and open, They really believe in their ambitious but naive vision. The technofascists are duplicitous. They know people won't buy their control fetish, so spend a lot more on presentation, sugar frosting, and outright lies about the "benefits of digital technology", and their own intentions.
Computers are neutral technology. It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. The computers belong to them now, they're just letting us use them. We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them: radiofrequency hardware firmware, digital rights management, encryption regulation. If left unchecked, the power of computing will cause massive damage to the entire intellectual property industry as well as many government functions such as law enforcement. Computing freedom is therefore against their agendas: they want to transform computers into tools of oppression in order to preserve the status quo or change it in their favor. We can buy iPhones but they're not really ours, they still belong to Apple, the software is under Apple's control so when they suddenly decide to oppress us with client side scanning there's nothing we can do.