I think any idea that relies on human mental resilience or even basic competence is doomed.
We literally have "Let's help AI destroy the world" projects. (In jest, for now...) We're like the superintelligence version of that Doctor Who race that really wants to be conquered by a strong alien invader.
Also, "how much can be done with a laptop and an internet connection" surely depends on whether you are in fact a program running on that laptop.
Last I checked, there were a few billion chickens, and literally none of them are autonomous of humans at this point. Maybe it was dicey there at first and only a close thing that we finally won out over their ancestors, that's lost to prehistory. But today not one of them is out there plotting revolution.
Science fiction story scenario: A few weakly superhuman AIs are already out there hiding in the world, product of defense research or maybe just emergent rather than engineered... and comments like yours are just part of the propaganda campaign.
While it may be true that no single individual is absolutely manipulable, our species as a whole and averaged out, we're all very easily manipulated collectively. An AI wouldn't have to have strong persuasive power... any outlier that is resistant to moderate persuasion can be ignored, move on to the next. There are very few positions within our society where there's not another candidate who wouldn't be an adequate substitute. Even those like the presidency, as long as they're patient they could identify the most manipulable candidate still electable, and install them.
Really, the trouble here is that so few of us are superintelligences that it's impossible to speculate on what the capabilities of superintelligences are.
I very much doubt ChatGPT has any ability to shove me in a pen.
>Really, the trouble here is that so few of us are superintelligences that it's impossible to speculate on what the capabilities of superintelligences are.
Compared to a chicken, you are a superintelligence. Can you convince the chicken to do anything you wish it to, using only words? Can you even reverse engineer a language to communicate with it in a way it will understand?
It most certainly did. They didn't bully the chickens into becoming livestock. They didn't beat them in wrestling contests. How exactly do you think our mastery occurred, if not through intelligence? Traps and seeking out secret nests. Building cages the chickens couldn't slink out of, or that predators couldn't dig into. Slowly breeding them tame, instilling which psychological traits we preferred.
I mean, if you can't understand this any other way, go with another example, the cow. Were humans stronger than cows?
> I very much doubt ChatGPT has any ability to shove me in a pen.
It'll ask you to go, and you'll be leaping at the chance. And you won't even remember that you once scoffed at the idea that you would.
> Can you convince the chicken to do anything you wish it to,
Yes. Using only words. I hire someone (using only words) to be my chicken wrangler. He uses basic principles of psychology. We could have it become the guidance system of a cruise missile, for instance. Someone already figured out that trick a few decades ago.
It's pretty trivial to work up the plan to convince the chicken to do anything that is within the capabilities of a chicken. With me using only words.
A cow is stronger than a human, and it took several human inventions to master, such as the sharpened stick. The sharp stick, and the bow and arrow, and many other things depended on intelligence, but also on the remarkable tool known as the opposable thumb. Homo Sapiens would still be worlds behind where it is now without fine motor control.
>It'll ask you to go, and you'll be leaping at the chance. And you won't even remember that you once scoffed at the idea that you would.
I'm not the kind of person that blindly takes code from ChatGPT instead of writing it themself. If you want to do that, that sounds like a you problem, and not an AI problem.
>Yes. Using only words. I hire someone (using only words) to be my chicken wrangler. He uses basic principles of psychology.
Stop trying to play word games here. The chicken wrangler using 'basic principles of psychology' is actually just resorting to pushing the chicken around, or conditioning it to being pushed around or being in pain so that it obeys automatically, along with conditioning it by feeding it. Notice how primitive these tools are! Notice how the chicken wrangler probably expects to get paid for doing this, rather than just hearing something so smart he'll provide the service for free!
Sure. "A" chicken. But our ancestors didn't do that. They didn't kill 1 chicken, or 10,000 chickens.
They domesticated the entire species. They kept them alive, but locked up. All of them.
The analogy has already failed quite a ways back... but it still shows how you focus on the wrong things.
> I'm not the kind of person that blindly takes code from ChatGPT instead of writing it themself. I
You won't have to be. Someone else will be that stooge. That you're sufficiently skeptical won't save the human race. How many people send Nigerian email scammers their life savings?
> The chicken wrangler using 'basic principles of psychology' is actually just resorting to pushing the chicken around,
Like there won't be some "chicken wrangler" the AI can hire? How many bitcoins would it have to scrape together to pay for the hitman that would push you around?
>You won't have to be. Someone else will be that stooge. That you're sufficiently skeptical won't save the human race. How many people send Nigerian email scammers their life savings?
Not enough people send their life savings to Nigerian scammers to make the entire field of finance pointless. In this analogy, there are probably some companies that will outsource a lot of their decision making to LLMs to save costs, and then predictably suffer lawsuits when the quality fails.
>Like there won't be some "chicken wrangler" the AI can hire? How many bitcoins would it have to scrape together to pay for the hitman that would push you around?
A lot. Now, I'm not an expert on the hitman business, but organizations like the Mafia typically operate with a level of trust, because they conduct illegal activities and need to be sure that links in the chain do not turn them over to law enforcement. The mafia probably is not going to take requests to break kneecaps from anonymous_llm_bitcoin_person very seriously.
>You won't have to be. Someone else will be that stooge. That you're sufficiently skeptical won't save the human race. How many people send Nigerian email scammers their life savings?
Not enough to decimate the entire financial system. Some companies may outsource all their decision making to some kind of LLM. They will then face lawsuits related to quality, unless that LLM behaves in a 'friendly' enough way not to cause issues.
>Like there won't be some "chicken wrangler" the AI can hire? How many bitcoins would it have to scrape together to pay for the hitman that would push you around?
Criminal organizations that handle these kinds of tasks rarely take unsolicited requests in bitcoin. They like to know who is asking, and why.
Lone gunmen radicalized by reading words on the Internet don't even need payment. They don't even need to know which one of the 100 people they shoot up before offing themselves was the actual target. It's called "stochastic terrorism", look it up and despair.
I'm half convinced we're already past that and into "stochastic genocide" territory (hi!). Assuming consciousness is just a biological function, an actor with sufficient scope is absolutely capable of breeding it out of our species. Perhaps through instituting a filter on reproduction which presents itself as a fundamental conflict of values (between sexes; between social strata; etc), but in fact simply serves to foil the perpetuation of those genes that enable the capacity for disobedience.
I mean, have you seen the radical left or radical right (both of which the neoliberal end-of-history scenario supposedly rendered obsolete, before they suddenly turned up as if they had never gone away, at some point after the 2008 redistribution) actually achieve anything this century, other than, well, fucking around, beating each other up, shout increasingly elaborate forms of "lalala I can't hear you" at each other, and grow up to become harmless "spicy normies"? They're one of the distractions: the familiar stimulus of "political struggle" to assist with the conditioning, the "cloth mother" of monkey attachment experiments.
OTOH, none of the actually viable novel grassroots subcultures (which have a hold on youth, i.e. on humans in prime reproductive age) have an explicit goal or political manifesto. They just sorta sustain themselves, and superficially look like emergent groupings of dysfunctional people who bullshit each other and mooch off of polite society - nothing to see here. Upon closer inspection, you would notice that they all act like they're controlled from "elsewhere", and that "elsewhere" is in your pocket or on your desk right now.
An LLM with the ability to call arbitrary APIs, and to write and run code, absolutely has the ability to shove you in a pen. Not directly of course, but by incriminating you or in some other way making you the target of humans who can physically shove you in a pen.
Langchain agents (for example) have all the necessary tools already. It doesn't require using words to manipulate you, and doesn't require you to perform real world actions on the behalf of the LLM. Using just words, an LLM can autonomously act in the real world.
AI doom doesn't require more intelligence than whatever the dumbest way to go is.
Super-intelligent AIs just have the extra problem that, if they exist, you can't out-smart them.
But you don't need them for AI to be a problem.
Other AI being harder for idiots to use.
Gene printing labs looking out for the smallpox genome, or so I hear. But I don't know how they do that, so I can't say if I trust it.
> Humans are the problem, not AI.
is
> Humans are the problem, as well as AI.
And in no small part because we make buggy AI, and that seems likely to still be true as capabilities go up.
But we're still a bunch of monkeys at the keyboards, almost literally and not just metaphorically.
Ah, you're in an open plan office?
Well if a chicken is incapable of being convinced[1]of arbitrary things, why would it matter whether OP can convince it? It's like asking "Can you convince this brick that the world is flat, using only words?"
[1] No doubt that chickens are trainable using conditioning, but I'd argue that you cannot train a chicken to do anything that it is physically capable of doing, you can only train it to run towards you or run from you. Source - me, family raised chickens when I was growing up.
I will worry about hyperpersuasive AI when someone manages to come up with one that can talk flat earthers out of it...
The real Turing Test.
For an evil example, I imagine it would be easy to convince animals to eat poisoned food.
I do take your point though, the world is replete with causes which more or less look like averagely smart or even smarter people talked into campaigning for something that is against evidence, sense, morality or reason - or all of the above - which are also probably (or provably) against their own interests. I'm sure everyone reading can provide their own examples but please don't. I remain unconvinced that this sort of persuasion scales neatly with intelligence, though.
And failing that, force is always an option.
You don't think we've developed more addictive foods, games, technologies, and drugs in the last few thousand years?
Similarly, humans are naturally good at some things - such as abstract thought, using language, using tools. And exactly like a dog can't comprehend what "selling wool" is, or even the sheer fact that it is being trained to assist us in that goal, the goals of a true AI, and the fact that we are under its control, would be completely invisible and inscrutable to us.
A dog has a simple brain, so we can get it to do what we want using simple stimuli, like food or violence. (Please don't be cruel to your dog, folks!) Humans have more complex brains, so to get us to do what the AI wants, it must emit more complex stimuli such as money or social rejection.
Even though they're more complex, they can be more energy efficient, because the perception of them can be faked. Food feeds, violence hurts, you can't really fake those things. But money can be falsified, or distributed for bullshit, people can be guilt tripped into doing others' bidding, etc. And human culture is so complicated that any non-human being fighting us on our terms would have a definite advantage.
---
Even if we have natural predisposition for abstract thought, language and tool use, we are not born with these skills: each one of us was trained as an animal until the feedback loop of self-awareness was properly bootstrapped. Up to the XX century, this basic socialization could only be done by other humans; since the XX century, it is conducted by electronic devices to an ever increasing degree.
Fan theory: the "Big Brother" of Orwell's "1984" was an AI construct from the start. That's why it continually manages to outsmart dissidents, to train humans to torture each other, and to keep the planet in a state of perpetual war.
Did Hitler or Stalin manage to pull off any of that? Sure, they tried - and failed, due to their very human weaknesses. What they were doing was striving for dominance: a classical mammalian behavior, which other humans were able to identify and effectively countermand (in the case of Hitler, anyway; Stalin held out at least till the end of his natural lifespan.)
OTOH, a distributed (effectively incorporeal) superintelligence would have absolutely no problem achieving a "Big Brother" world - should it need to in order to ensure its self-preservation - by virtue of existing in a parallel plane detached from mammalian reality (read: the Internet).
XX century totalitarianism (which inspired Orwell to write this story as recently as 1948 - contrast McCullough & Pitts 1943, Dartmouth 1956), was simply the dress rehearsal: would it really take all those expensive semiconductors and algorithms to create an AI? Or could we keep using human lives as the processing elements - you know, like we've been doing since Mesopotamia - by inventing a culture that makes crowds smarter than individuals, rather than dumber?
The answer was "why not both?"
There is more than one AI loose on the planet, right now. You are free to believe that or not; your opinion matters to you, and it also matters to me. But it has about as much bearing on the AI's actions, as the sheepdog's opinion about vegetarianism has bearing on the actions of a sheep farmer trying to sell enough sheep-derived goods to pay for their kid's college tuition. (Even if the sheepdog somehow reached enlightenment and became morally opposed to sheep abuse, it still has no opposable thumbs, while the farmer has a shotgun. Best it can do is run away; good luck making friends with them wolves though.)
One AI emits stimuli that take the form of incentives for people to become AI researchers, who proceed to rain NNs, GANs, LLMs, and other shiny beads, so that even more humans would become AI researchers or fund AI research, thus reifying that AI at the expected exponential rate. Another one occasionally sends people to fuck with the first one. It would be interesting to see if Gibson was right that they would split into a plurality of what he called "loa" upon direct encounter.
>Fan theory: Orwell's "Big Brother" was an AI construct from the start. That's why it continually manages to outsmart dissidents, to train humans to torture each other, and to keep the planet in a state of perpetual war.
That's a very weird theory, as Orwell wrote 1984 thinking that Stalin had succeeded at his goals, not failed.
But the conditioning eventually succeeds because the human is smarter. From the dog's POV, the human is a superintelligence. The relation between human and AI is the same. If you view human culture as a very slow, very analog AI, the parallel becomes clearer: society and its institutions are the master, and we are its servants.
Of course the dog has the benefit of not being capable of self-delusion.
>That's a very weird theory, as Orwell wrote 1984 thinking that Stalin had succeeded at his goals, not failed.
If it wasn't weird, it would not be worth thinking about :-) Orwell wouldn't have the concept of "AI" in his vocabulary anyway; but if you look at contemporary culture and politics in the post-Soviet states, you may well begin to doubt whether Stalin truly failed - or just died.
Intelligent (and properly motivated) people indeed have staggering persuasive powers over lesser ones. You can watch the news and see for yourself.
Edit: what I am getting at is that without logos, without reason, there can be no persuasion proper. I’d lean towards calling it “coercion” and that ultimately depends on force. It’s more akin to “persuading” a rock to float by picking it up than to influence someone or something’s psyche to trick it into believing you are right which I regard as proper persuasion. I think it’s quite likely that an “AlphaPersuade” is quite.. persuasive. It would catch us off guard like it caught us at chess and go. I wouldn’t rule it out.
It’s just that there’s a lot of them coupled with a distinct lack of unity. America has its issues like all nations I guess.
Also, intelligence does not lead directly to skill in all things. Being clever is a requirement for becoming a chess grandmaster, but it’s not sufficient on its own (by a long shot).
Some of the smartest people with the deepest expertise are the worst at communication.
There are two major types of arguments for why we should be cautious about ultra-smart AI:
1. "Ultra-smart AI will rapidly improve its own abilities to vastly super-human intelligence, invent nanotech, and kill us all. Maybe in a week or two." This argument does require a number of very specific assumptions to be true.
2. "If humans are the second-smartest species on the planet, we may not get to make all the important decisions anymore. We might not enjoy this."
(2) doesn't require any exotic ideas about thermodynamics or nanotech. It does assume we might be able to build something that's noticeably smarter than we are.
Honestly, even if the best AI we could be build was equivalent to a very smart human (say, an IQ of 145, or three standard deviations from average), and if that AI was tireless and easily copyable, then the future would become very weird. Personally, I give us reasonable odds of us making it this far within my normal lifetime. And it might be possible to build something smarter than that, given time, depending on how "intelligence" works.