News for hackers. Stuff that matters.
Re:Universal Paperclips by CharlesW on Wednesday August 02, 2023 @01:00PM (#36973574)
I wonder if he got the idea from Philip K. Dick's 1955 story "Autofac"?

https://en.wikipedia.org/wiki/Autofac

https://www.vulture.com/2018/01/electric-dreams-recap-season...

Re:Universal Paperclips by actionfromafar on Wednesday August 02, 2023 @01:04PM (#36973631)
PKD keeps surprising me.
Re:Universal Paperclips by dylan604 on Wednesday August 02, 2023 @01:09PM (#36973700)
I like how the sci-fi authors have spent time thinking about what an advanced AI could do, yet those building the AI have not taken a moment's pause to consider what they are doing
Re:Universal Paperclips by d1str0 on Wednesday August 02, 2023 @01:14PM (#36973780)
Do you honestly believe that the only people who have thought about the ramifications of AI are Sci Fi authors? I can guarantee people who spend years researching and building advanced language models have thought about the ramifications of their work.

This isn’t Jurassic Park.

Re:Universal Paperclips by actionfromafar on Wednesday August 02, 2023 @01:53PM (#36974382)
Haha, but it is! :-D

LLMs are pretty basic stuff but we are all struggling with what to use them for!

OpenAI is manually playing whac-a-mole with ChatGPT saying the darndest things!

Re:Universal Paperclips by otachack on Wednesday August 02, 2023 @01:54PM (#36974408)
Depends what we're looking at. Ride share disruption was very Jurassic Park and we've been dealing with ramifications ever since.
Re:Universal Paperclips by AlotOfReading on Wednesday August 02, 2023 @01:57PM (#36974457)
If you accept the implied premise that there are irresponsible deployments of AI out there, the alternative explanation is that they did consider the ramifications and simply don't care. That's even worse. Calling them ignorant is actually giving them the benefit of the doubt.
Re:Universal Paperclips by goatlover on Wednesday August 02, 2023 @02:14PM (#36974685)
Or the researchers don't think existential threats are realistic, and paper maximizing thought experiments are silly. Maybe they're wrong, but maybe not. It's easy to imagine AI takeover scenarios by giving them unlimited powers, it's hard to show the actual path to such abilities.

It's also hard to understand why an AI smart enough to paperclip the world wouldn't also be smart enough to realize the futility in doing so. So while alignment remains an issue, the existential alignment threats are too ill-specified. AGIs would understand we don't want to paperclip the world.

Fun game though.

Re:Universal Paperclips by AlotOfReading on Wednesday August 02, 2023 @02:39PM (#36975073)
That would be not accepting the premise that deployments are irresponsible. I guess there could be a situation where every researcher thinks everyone else's deployment is irresponsible and theirs is fine, but I don't think that's what you're saying.
Re:Universal Paperclips by cubefox on Wednesday August 02, 2023 @03:00PM (#36975440)
> It's also hard to understand why an AI smart enough to paperclip the world wouldn't also be smart enough to realize the futility in doing so.

Terminal goals can't be futile, since they do not serve to achieve other (instrumental) goals. Compare: Humans like to have protected sex, watch movies, eat ice cream, even though these activities might be called "futile" or "useless" (by someone who doesn't have those goals) as they don't serve any further purpose. But criticizing terminal goals for not being instrumentally useful is a category error. For a paperclipper, us having sex would seem just as futile as creating paperclips seems to us. Increased intelligence won't let you abandon any of your terminal goals, since they do not depend on your intelligence, unlike instrumental goals.

Re:Universal Paperclips by Sardtok on Wednesday August 02, 2023 @04:21PM (#36976785)
It's not like you want to eat ice cream constantly, even if it means making everything into ice cream.

Of course the premise becomes that the AI has been instructed to make paperclips. They should have hired a better prompt engineer, capable of actually specifying the goals more clearly. I don't think an AI that eradicates humankind, will have such simplistic goals, if an AI ever becomes the end of humans. Cybermen, though, are inevitable.

Re:Universal Paperclips by sfink on Wednesday August 02, 2023 @06:15PM (#36978487)
Yes, they should just write prompts without bugs. Can't be that much harder than writing software without bugs.
Re:Universal Paperclips by ASalazarMX on Wednesday August 02, 2023 @04:27PM (#36976870)
> AGIs would understand we don't want to paperclip the world.

Even if they did, what if they aren't smart enough for eloquent humans to convince them it's for the greater good. True AGIs will need a moral code to match their intelligence, and someone will have to decide what's good and bad to make that moral code.

Re:Universal Paperclips by goatlover on Wednesday August 02, 2023 @06:33PM (#36978721)
Then they won't be smart enough to paperclip the world. No human organization can do that.
Re:Universal Paperclips by ASalazarMX on Thursday August 03, 2023 @05:43PM (#36992403)
I've seen people calculate how much human blood would be needed to make an iron sword, for fun. AGIs won't need the capability to transmute all matter into iron, just enough capabilities to become significantly dangerous.
Re:Universal Paperclips by mecsred on Wednesday August 02, 2023 @04:41PM (#36977082)
I agree completely with your first paragraph, and disagree completely with your second.

"Futility" is subjective, and the whole purpose of the thought experiment is to point out that our predication of "futility" or really any other purely mental construct does not become automatically inherited by a mind we create. These imaginary arbitrarily powerful AIs would definitely be able to model a human being describing something as futile. Whether or not it persues that objective has nothing to do with it understanding what we do or don't want.

Re:Universal Paperclips by dylan604 on Wednesday August 02, 2023 @03:38PM (#36976060)
See also social media platforms. They are very well informed of the results of their algorithmic changes.

See also big tobacco. They exactly what their additives to the product did.

See also 3M and PFAS. See also Big Oil. See also, see also...

Why would I expect anything different from any other branch of business using the precedence laid before us?

Re:Universal Paperclips by FrustratedMonky on Wednesday August 02, 2023 @03:47PM (#36976225)
I think they do know. Corporations are filled with people that 'know' but can't risk leaving, so they comply, and even promote such decisions. It's a form of group think with added risk of being fired, passed over for promotion.

Eichmann.

Re:Universal Paperclips by mikea1 on Wednesday August 02, 2023 @04:41PM (#36977067)
Another explanation is that there are those who considered and thoughtfully weighed the ramifications, but came to a different conclusion. It is unfair to assume a decision process was agnostic to harm or plain ignorant.

For example, perhaps the lesser-evil argument played a role in the decision process: would a world where deep fakes are ubiquitous and well-known by the public be better than a world where deep fakes have a potent impact because they are generated seldomly and strategically by a handful of (nefarious) state sponsors?

Re:Universal Paperclips by parpfish on Wednesday August 02, 2023 @06:02PM (#36978317)
there's also the issue that most of the AI catastrophizing is a pretty clear slipperyslope argument:

if we build ai AND THEN we give it a stupid goal to optimize AND THEN we give it unlimited control over its environment, something bad will happen.

the conclusion is always "building AI is wrong" and not "giving AI unrestricted control of critical systems is wrong"

Re:Universal Paperclips by mikea1 on Wednesday August 02, 2023 @06:21PM (#36978563)
I completely agree that's a valid argument. I just think it is rational for someone to come to a different conclusion, given identical priors.
Re:Universal Paperclips by parpfish on Wednesday August 02, 2023 @06:45PM (#36978851)
If it wasn’t clear, I agree with your parent comment
Re:Universal Paperclips by mikea1 on Wednesday August 02, 2023 @08:04PM (#36979593)
My main takeaway from Bostrom's Superintelligence is that a super intelligent AI cannot be contained. So, the slippery slope argument, often derided as a bad form of logic, kind of holds up here.
Re:Universal Paperclips by olddustytrail on Wednesday August 02, 2023 @06:57PM (#36979001)
The massive flaw in your argument is your failure to define "we".

Replace the word "we" with "a psychotic group of terrorists" in your post and see how it reads.

Re:Universal Paperclips by actionfromafar on Wednesday August 02, 2023 @08:08PM (#36979621)
Or "board of directors beholden to share-holders".
Re:Universal Paperclips by parpfish on Wednesday August 02, 2023 @08:29PM (#36979784)
If you’re talking about some group of evildoers that deploy ai in a critical system to do evil… the issue is why do they have control to the critical system? Surely they could jump straight to their evil plot with the ai at all
Re:Universal Paperclips by olddustytrail on Thursday August 03, 2023 @03:45PM (#36990801)
Your question is equivalent to "if you have access to the chessboard anyway, why use Stockfish, just play the moves yourself."
Re:Universal Paperclips by chess_buster on Wednesday August 02, 2023 @01:58PM (#36974477)
They have – and decided to do it anyway.
Re:Universal Paperclips by throwaway290 on Wednesday August 02, 2023 @02:05PM (#36974573)
> I can guarantee people who spend years researching and building advanced language models have thought about the ramifications of their work.

Super easy to not think about something if your job depends on it. And even if you do, things don't go as you think (see bombings of civilian Hiroshima and Nagasaki despite objections of nuclear physicists).

Re:Universal Paperclips by mitthrowaway2 on Wednesday August 02, 2023 @02:22PM (#36974791)
Some of us considered it and even decided to go into different fields as a result.

Some others entered the field, made progress, and apparently regretted it.

Others are willing to put their concerns aside for money. Salaries get very high in that field.

Re:Universal Paperclips by killerstorm on Wednesday August 02, 2023 @05:11PM (#36977541)
That's not how it works. People publish papers demonstrating improvements without thinking about "ramifications".
Re:Universal Paperclips by JRandomHacker42 on Wednesday August 02, 2023 @02:01PM (#36974511)
https://twitter.com/AlexBlechman/status/1457842724128833538

Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale

Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus

Re:Universal Paperclips by willcipriano on Wednesday August 02, 2023 @02:32PM (#36974954)
The people building it know it isn't AI, the people selling it call it that.
Re:Universal Paperclips by Karunamon on Wednesday August 02, 2023 @06:46PM (#36978871)
To say that something like GPT-4 does not count as "AI" requires a gargantuan shifting of goalposts from where they were at this time last year.
Re:Universal Paperclips by actionfromafar on Wednesday August 02, 2023 @08:09PM (#36979634)
So it was ever since computers started playing chess.
Re:Universal Paperclips by feoren on Wednesday August 02, 2023 @06:18PM (#36978522)
That's because the people who are building the AI actually know how it works, understand how fundamentally simple it all is, and know that there's no room for consciousness to magically emerge. The current state of AI is not so much a story of any kind of "intelligence" being amazing, but rather the sum total of humanity's data being amazing. The amazing feats LLMs perform come from the words we all wrote, not the code they wrote. The code just unlocks the previously latent power of all that data.

It is nothing close to being an actual intelligence, regardless of how much we anthropomorphize it. We also anthropomorphize stick figures, stuffed animals, and weighted companion cubes.

Re:Universal Paperclips by bananapub on Wednesday August 02, 2023 @06:44PM (#36978846)
that's indeed not a sensible worry, but the actual consequences on society of such things are extremely real and already happening, and are something the people involved seem either uninterested in worrying about or actively encouraging.
Re:Universal Paperclips by flangola7 on Wednesday August 02, 2023 @07:04PM (#36979068)
You've written a lot of words yet I don't see a compelling reason to believe a single one of them.
Re:Universal Paperclips by actionfromafar on Wednesday August 02, 2023 @07:05PM (#36979076)
Good thing no new technology ever caused anything bad thanks to us not anthropomorphising it.
Re:Universal Paperclips by abecedarius on Wednesday August 02, 2023 @02:42PM (#36975125)
He got it from Eliezer Yudkowsky's somewhat different paperclip maximizer in a mailing-list post. (My memory of a Twitter thread a while back which included Yudkowsky, saying he'd told Bostrom not to worry about attributing ideas like that to him.)
Re:Universal Paperclips by davidmnoll on Wednesday August 02, 2023 @03:32PM (#36975968)
Is that a joke?