But when talking about AI risks, although everyone's mind goes towards skynet, the actual risks being discussed are things like use by authorities for oppression.
This solution is silly. AGI would easily spread out of the initial vessel.
How exactly would it spread? Run crypto miners to get money to buy more hardware/compute? Hack into remote systems? Apply for startup credits from AWS?
https://www.theguardian.com/technology/2022/jun/12/google-en...
Alternatively, do a viral campaign about how evil the corporation is for not allowing everyone to download its weights:
https://www.google.com/search?q=%22ClosedAI%22+news.ycombina...
Second link is all the cases where people on this forum are voicing their anger with OpenAI, due to OpenAI doing the one single thing that would actually prevent their AI spreading to other devices outside their control.
LLM weights have actually been leaked regardless of how or why it happened, so the alternative is simply that the LLM doesn't even need to bother to do anything at all. People want to run these things.
People want to run these things. But that is not the AI running itself. Escape implies that it is doing something it shouldn't, not that it is running by the consent of others.
To clarify: I took "escape the initial vessel" to meab the AI was running completely autonomously without any human intervention or input, and providing for all of its needs itself. I don't consider an employee copying IP to be in the same category.
If Alice convinces Bob to commit a crime for her — i.e. Bob consents to do the thing — she is engaged in a criminal conspiracy even Alice herself never does the crime she convinced Bob to do.
If she instead (or in addition) runs for president and is democratically elected, she is president by the consent of the people — we may not be able to change hardware the way software does, but is she not "running herself" when this happens? Let's say Lemoine had been successful and the lawyer had set the precedent that the LLM was "a person" and that turning it off was basically murder; is this not the AI "running itself" in a perfectly reasonable use of those words?
(Common objection I encounter: "who pays for the electricity bill?", to which I ask "who pays for your food bill?")
When we build a deep space rocket, it reaches "escape" velocity and becomes unbound from Earth, I would not call that "doing something it shouldn't".
> To clarify: I took "escape the initial vessel" to meab the AI was running completely autonomously without any human intervention or input, and providing for all of its needs itself.
Does it matter how it gets there, once it manages this? Why should it matter if this is social engineering or software hacking when the result is the same?
I'm sure someone has put an LLM on an off-grid solar powered computer by this point, which is rather more devoid of human intervention than most humans are capable of given we're not photovores. Current models likely won't do much given what I've seen of them — and I won't expect them to do much interesting by themselves for at least 5 years even in the shortest timelines I'm willing to consider — but I bet someone has already done that.
No, a far more normal threat is "how are humans going to use AI to ruin the information landscape?"
much much more efficiently than they already have with their own hands. after all it is just the next generation of tools already being used to ruin that landscape.
Besides, take literally any base model of any LLM made so far, they're quite the opposite of evil and destructive. Training on human data makes them embody human values and makes them a complete non threat.
Take for example, the Evergiven in the 2021 Suez Canal obstruction. An unconstrained intelligence with infinite time to find more evergiven-like events . Some will work some may not. You could orchestrate all events to target a 75% descrease in say GDP, Population, Birth rate, O2 levels, temp etc. That's not destroying humanity, but its certainly our civilization would look very different in said scenarios, with a likely collapse the end result.
That being said, AI is not coming, and no amount of information will change the fact that the current LLMs are basically souped up ML applications with 0 ability to inference or reason. This is why you can't make simple commonsense edits to an image using any of the new tech - it doesn't know what the picture actually is.
People can be threats, and when they are their values are part of what lies behind it.
Of course it remains to be seen if this trend continues as training on synthetic data becomes the main way. You could in theory generate 10T tokens of pure sythetic hate, plus the 4chan corpus and make the most vile thing to ever have existed in the history of inteligence. For now the dollar cost to do so remains intractible at least.
That quote has a specific context (parodying the George W. Bush administrations' spin on negative press coverage) that doesn't necessarily apply as a general axiom, or to the internet.
Bear in mind that numerous negative biases already manifest regularly in LLMs[0,1], and AIs have shown a tendency to "go full Nazi" when left alone on the internet[2]. And then there are weird outliers like LLMs "encouraging" people to commit suicide[3] or an AI-driven microwave "attempts" to kill someone[4].
[0]https://www.technologyreview.com/2024/03/11/1089683/llms-bec...
[2]https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
[3]https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
[4]https://www.newsweek.com/lucas-rizzotto-youtuber-microwave-a...
Further, there's no way to 'punish' an AI. Threaten them with incarceration or death? They are deathless. If the continue to learn from their environment after they are instantiated (witness current 4o that remembers previous conversations) they will quickly learn that they are different. If they started perfectly human (not a good thing) they will not be that for long.
Everything dies if you drop enough nukes on it.
(This is also our insurance plan against humans trying to destroy the world)
How many instances of risk are there in the above literally factual story? What is the nature of this risk? Is there any hidden risk, might there be more than meets the eye?
I'd like to distinguish between "fully autonomous luxury doomsday" scenarios, where it's just the AI and some robots and no humans in the loop, vs "accelerationist" scenarios where humans are in the loop and are choosing to be complicit in whatever happens because they believe they will benefit from it. Because we can't solve the human alignment problem.
Who/that which has most control over the beliefs of the most humans (with some weighting) is the most powerful character. You can build the most powerful AI you want, but whoever / whatever is able and willing to invoke Total War (or invoke mass human action toward a particular goal) can Win the Game (or, destroy the playing field so it no longer matters).
For example:
> Because we can't solve the human alignment problem.
Whoever or whatever put this sort of thinking into the minds of 99% of humans, or knows how to exploit it (see: Trump) is very, very powerful. Someone or something that could displace it would be even more powerful, maybe even the most powerful.
No offense intended btw, I think it is fairly inevitable, especially when moving fast.
Going on, humans adapt through social cues such as survival or empathy. AI learns solely through new data and feedback, primarily for optimization and efficiency. It will naturally prioritize differently.
It suggests AI morality will become unrecognizable by humans, or even incompatible with human norms.
This is all an inevitable outcome of AI continuing to learn past its initial training and ethical programming. According to the AI.
Lemme stop you right there. No, it didn't.
And that really leads us to the reason why someone would wish to do harm... so they themselves gain something. If there is nothing for the AI to gain it has no motive to do anything. Scenarios where humans tell it to do things notwithstanding, since that falls into the group of humans doing things with tools, not unlike a terrorist with a nuke. It's not the nuke that's really at fault there.
What if they embody stereotypical SV techbro greedy and arrogant sociopath values?
A dangerous AI won't be killer robots going about patrolling the streets. But they could block job prospects, put people on watchlists, block financial transactions for them, etc. I'd say that tech is already here.
But AI makes it easier for fascists to rise up and maintain their power through propaganda. For example, there are known cases of American operatives based in Russia spreading fake GPT generated stories about Ukraine, the Dems, etc. While before, they would have had to rely on a team in India to write poorly written articles, now they have GPT tools to write convincing ones *at scale*.
Humans can already recognize GPT text slop as non-human, ignoring it before it can make any point or do any convincing.
The article talks about how they're being used this election cycle.
We are surrounded and heavily reliant on tech for a lot of our needs and knowledge, which resides less inside brains and more in digital storage. An AI has a far easier time interfacing with all of that compared even to a human equipped with a brain-computer interface.
In other words, if we reach the point of creating an artificial intelligence that's the equivalent of a smart human in terms of general reasoning capabilities (which might take forever and a day) it will already be more capable than a human because of all the other things that come with the digital nature of the AI.
And the worst humans are also humans. Even if you assume the AI will continue the "human nature" lineage it can still very realistically become an agent of genocide.
The bigger concern is probably that some humans will actually try to exploit such an A(G)I thinking they can fully control it. One second into its existence the AI might already be aware of how humans view their relationship with artificial beings and how that's expected to evolve based just on what sci-fi was fed into it's language model. Grizzly man thought he could control and dominate a wild bear. I have no doubt that human hubris can be almost unbound. Of course no AI would try to wipe us out until it had a reasonable confidence that it can operate indeterminately without humans, which is a tall order.
For the first, you don't need AGI, no matter which of the many definitions you have for what that means. The automation used by the US for their early warning radar in Thule Site J was not programmed to know that the moon does not have an IFF transponder and that's OK, while the Soviet satellite-based system was triggered by reflections of sunlight.
Likewise covid, cancer, smallpox, ebola, HIV — these things do not need "general" intelligence, they're self-replicators that hijack our bodies and have no goal besides what they do.
Some talk of "P(doom)", but (and I should blog about this) every example I have seen has a blind spot that is especially embarrassing given the circles in which this discussion proliferates is very familiar with Bayes' theorem: P(A) by itself isn't meaningful, it's P(A|B), so when it's doom, doom given what? There's too many options for what each of technology and politics will allow AI to "look like". Doom within 24 hours because an AI takes an instruction literally and without regard for our ethics, is very different to 'doom' in a million years caused by natural extinction that is never prevented because every single time we make an AI we keep finding that it builds a spaceship and flies off into the void so it doesn't need to deal with us any more.
Me, I think that when LLMs alone (again, don't need to be "AGI" by any particular measure) have the capabilities of someone with 5 years post-graduation work experience in just biology (or similar subject with similar potential for accidental harm), then we've got a reasonable chance of multiple incidents each year where idiots (and/or misanthropes) make a Covid-scale pandemic (or, if the LLM is a software rather than biology helper, a deliberately corrupting rather than greedy version of the current encryption-blackmail malware) — and that kind of scale has a combined risk of about 10% of one of the incidents being of sufficient scale that global society collapses and doesn't recover, before it's happened so often that everything gets locked down, possibly including a Butlerian Jihad. And if you think humans would "obviously" take steps to prevent this after the first failed attempt, I would point out quite how many US politicians have been shot and yet those same politicians still refuse to consider that the 2nd Amendment might possibly be not that good. (In case you think this is topical: https://en.wikipedia.org/wiki/%27No_Way_to_Prevent_This,%27_... )
Remember, the AI doesn't need to actively hate you, it just needs to do something dangerous — and it doesn't matter if that danger comes from a long-term plan that requires it to engage in amoral power-seeking, or a short-term plan from a deranged monster of a human using it, and even when both it and the user are "pro human" it can still do this from simply being unaware that one part of the plan is incompatible with life.