News for hackers. Stuff that matters.
Re:AI’s big rift is like a religious schism by crotchfire on Wednesday December 13, 2023 @05:38AM (#38625386)
so as to be fundamentally irreconcilable with our understanding of agency, desire, intention, survival

Exactly.

Human have self-preservation drive because we evolved in an environment where, for billions of years, anything without a drive to reproduce and self-preserve ceased to exist.

Why should a gradient-descent optimizer have this property?

It's just absurd generalization from a sample size of one. Humans are the only thing we know of that's intelligent, humans seek to self-preserve, therefore everything intelligent must seek to self-preserve!

It's idiocy.

Re:AI’s big rift is like a religious schism by tim333 on Wednesday December 13, 2023 @07:26AM (#38626359)
>Why should a gradient-descent optimizer have this property?

Because some human may give it that. You may have noticed random code downloaded off the internet may do bad things, not because code and an inherent desire to be bad but because humans.

Re:AI’s big rift is like a religious schism by amelius on Wednesday December 13, 2023 @06:19AM (#38625722)
Because self-preservation is a desirable property for anyone wanting to build an AI army?
Re:AI’s big rift is like a religious schism by crotchfire on Wednesday December 13, 2023 @06:25AM (#38625774)
In fact it is not. Capacity for mutiny is not a selling point for defense contractors.