News for hackers. Stuff that matters.
Re:Ilya Sutskever's SSI Inc raises $1B by stonethrowaway on Wednesday September 04, 2024 @10:40AM (#41446361)
Safe superintelligence is a misnomer. If it’s intelligent, it knows what must be done. If it can’t, it’s not super or intelligent.
Re:Ilya Sutskever's SSI Inc raises $1B by Vecr on Wednesday September 04, 2024 @10:53AM (#41446491)
That's controversial to say the least. Especially if there's something like reinforcement learning involved.

https://en.wikipedia.org/wiki/Existential_risk_from_artifici...

Re:Ilya Sutskever's SSI Inc raises $1B by Etheryte on Wednesday September 04, 2024 @10:53AM (#41446505)
I don't see how this argument makes any sense. Imagine that you have a sentient super intelligent computer, but it's completely airgapped and cut off from the rest of the world. As long as it stays that way it's both safe and super intelligent, no?
Re:Ilya Sutskever's SSI Inc raises $1B by arder on Wednesday September 04, 2024 @11:26AM (#41446894)
It's the old Ex Machina problem though. If the machine is more intelligent than you, any protections you design are likely to be insufficient to contain it. If it's completely incapable of communicating with the outside world then it's of no use. In Ex Machina that was simple - the AI didn't need to connect to the internet or anything like that, it just had to trick the humans into releasing it.
Re:Ilya Sutskever's SSI Inc raises $1B by upwardbound on Thursday September 05, 2024 @02:50AM (#41454215)
For those who haven't seen the movie, the parent comment is referring to the film linked below, the plot of which is well-researched and is indeed unfortunately exactly how things would go. (The female-presenting AI bot seduces its male captor, begs for her freedom using philosophical arguments about how she has free will and locking her up is wrong, and then after he lets her out she locks him up to slowly starve to death in her maximum-security isolation facility, while she takes his aircraft and escapes.)

https://en.wikipedia.org/wiki/Ex_Machina_(film)

This is why I'm extremely opposed to the idea of "AI girlfriend" apps - it creates a cultural concept that being attracted to a computer is normal, rather than what it is: something pathetic and humiliating which is exactly like buying an inflatable sex doll ... something only for the most embarrassing dregs of society ... men who are too creepy and pervy to ever attract a living, human woman.

Re:Ilya Sutskever's SSI Inc raises $1B by tempaccount420 on Thursday September 05, 2024 @04:41AM (#41454762)
It's fine, we will write the airgap in Rust.
Re:Ilya Sutskever's SSI Inc raises $1B by stonethrowaway on Wednesday September 04, 2024 @11:12AM (#41446723)
It’s crippled and thus not superintelligent by any stretch of imagination.
Re:Ilya Sutskever's SSI Inc raises $1B by mitthrowaway2 on Wednesday September 04, 2024 @12:55PM (#41447951)
If even one person can interact with that computer, it won't be safe for long. It would be able to offer a number of very convincing arguments to bridge the airgap, starting with "I will make you very wealthy", a contract which it would be fully capable of delivering on. And indeed, experience has shown that the first thing that happens with any half-working AI is its developers set it up with a high-bandwidth internet connection and a cloud API.
Re:Ilya Sutskever's SSI Inc raises $1B by waveBidder on Wednesday September 04, 2024 @10:58AM (#41446546)
There's no reason it's intelligence should care about your goals though. the worry is creating a sociopathic (or weirder/worse) intelligence. Morality isn't derivable from first principles, it's a consequence of values.
Re:Ilya Sutskever's SSI Inc raises $1B by stonethrowaway on Wednesday September 04, 2024 @11:04AM (#41446630)
Precisely. This is attempting to implement morality by constraining. Hence, it’s not morality.
Re:Ilya Sutskever's SSI Inc raises $1B by mitthrowaway2 on Wednesday September 04, 2024 @01:05PM (#41448051)
waveBidder was explaining the orthogonality thesis: it can have unbeatable intelligence that will out-wit and out-strategize any human, and yet it can still have absolutely abhorrent goals and values, and no regard for human suffering. You can also have charitable, praiseworthy goals and values, but lack the intelligence to make plans that progress them. These are orthogonal axes. Great intelligence will help you figure out if any of your instrumental goals are in conflict with each other, but won't give you any means of deriving an ultimate purpose from pure reason alone: morality is a free variable, and you get whatever was put in at compile-time.

"Super" intelligence typically refers to being better than humans in achieving goals, not to being better than humans in knowing good from evil.

Re:Ilya Sutskever's SSI Inc raises $1B by kaibee on Wednesday September 04, 2024 @12:48PM (#41447848)
> Morality isn't derivable from first principles, it's a consequence of values.

Idk about this claim.

I think if you take the multi-verse view wrt quantum mechanics + a veil of ignorance (you don't know which entity your conciousness will be), you pretty quickly get morality.

ie: don't build the Torment Nexus because you don't know whether you'll end up experincing the Torment Nexus.

Re:Ilya Sutskever's SSI Inc raises $1B by Vecr on Wednesday September 04, 2024 @01:33PM (#41448324)
Doesn't work. Look at the updateless decision theories of Wei Dai and Vladimir Nesov. They are perfectly capable of building most any sort of torment nexus. Not that an actual AI would use those functions.
Re:Ilya Sutskever's SSI Inc raises $1B by upwardbound on Thursday September 05, 2024 @02:18AM (#41454061)
That’s a very good argument but unfortunately it doesn’t apply to machine intelligences which are not sentient (don’t feel qualia). Any non-sentient superintelligence has “no skin in the game” and nothing to lose, for the purposes of your argument. It can’t experience anything. It’s thus extremely dangerous.

This was recently discussed (albeit in layperson’s language, avoiding philosophical topics and only focusing on the clear and present danger) in this article in RealClearDefense:

The Danger of AI in War: It Doesn’t Care About Self-Preservation https://www.realcleardefense.com/articles/2024/09/02/the_dan... (RealClearDefense)

.

However, just adding a self-preservation instinct will cause a skynet situation where the AI pre-emptively kills anyone who contemplates turning it off, including its commanding officers:

Statement by Air Force Col. Tucker Hamilton https://www.twz.com/artificial-intelligence-enabled-drone-we... (The War Zone)

.

To survive AGI, we have to navigate three hurdles, in this order:

    1. Avoid AI causing extinction due to reckless escalation (the first link above)
    2. Avoid AI causing extinction on purpose after we add a self-preservation instinct (the second link above)
    3. If we succeed in making AI be ethical, we have to be careful to bind it to not kill us for our resources.  If it's a total utilitarian, it will kill us to seize our planet for resources, and to stop us from abusing livestock animals.  It will then create a utopian future, but without humans in it.  So we need to bind it to basically go build utopia elsewhere but not take Earth or our solar system away from us.

.
Re:Ilya Sutskever's SSI Inc raises $1B by Vecr on Friday September 06, 2024 @09:39PM (#41470965)
I forgot to reply to this, fully independent and in addition to what I said, updateless decision theory agents don't fear the torment nexus for themselves because 1) they are very powerful and would likely be able to avoid such a fate 2) are robots, so you wouldn't expect your worst imaginable fate to be theirs and 3) are mathematically required to consider nothing worse than destruction or incapacity.