News for hackers. Stuff that matters.
Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by wruza on Saturday May 18, 2024 @01:12AM (#40396582)
AGI started blending people to extract iron from their blood to produce more paperclips

That’s neither efficient nor optimized, just a bogeyman for “doesn’t work”.

Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by FeepingCreature on Saturday May 18, 2024 @06:52AM (#40397907)
You're imagining a baseline of reasonableness. Humans have competing preferences, we never just want "one thing", and as a social species we always at least _somewhat_ value the opinions of those around us. The point is to imagine a system that values humans at zero: not positive, not negative.
Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by freehorse on Saturday May 18, 2024 @10:01AM (#40399165)
Still there are much more efficient ways to extract iron than from human blood. If that was the case humans would have already used this technique to extract iron from the blood of other animals.
Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by FeepingCreature on Saturday May 18, 2024 @10:10AM (#40399213)
However, eventually those sources will already be paperclips.
Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by freehorse on Saturday May 18, 2024 @11:15AM (#40399705)
We will probably have died first by whatever disasters the extreme iron extraction on the planet will bring (eg getting iron from the planet's core).

Of course destroying the planet to get iron from its core is not a popular agi-doomer analogy, as that sounds a bit too human-like behaviour.

Re:Ex-OpenAI staff must sign lifetime no-criticism contract ... by FeepingCreature on Saturday May 18, 2024 @02:47PM (#40401084)
As a doomer, I think that's a bad analogy because I want it to happen if we succeed at aligned AGI. It's not doom behavior, it's just correct behavior.

Of course, I hope to be uploaded to the WIP dyson swarm around the sun at this point.

(Doomers are, broadly, singularitarians who went "wait, hold on actually.")