News for hackers. Stuff that matters.
Re:Evolution of AI and Amara's Law by nunocoracao on Monday January 22, 2024 @04:01PM (#39095381)
Agree, the path might be quite bumpy but so could the endgame. Whether AGI wants to help or kill the human race will depend on how aligned it is to our goals. Now, since "our" goals will depend on whoever has the metaphysical ear of such AI this could be problematic.
Re:Evolution of AI and Amara's Law by visarga on Monday January 22, 2024 @05:17PM (#39096286)
I think I read this kind of discussions too many times, imagine how the training set of GPT-5 will look like? all sorts of theories of "whether AGI wants to help or kill the human race" spread over the internet and analyzed to death. It is going to know this topic in-and-out. Will be able to write a masterful dissertation on the topic.
Re:Evolution of AI and Amara's Law by ben_w on Tuesday January 23, 2024 @03:09AM (#39100727)
Yes, and that's good because GPT-5 is going to be coming from a team that cares about alignment and which has values only a little different from mine. They can still mess up: https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt

OTOH, an LLM trained by the FSB or CIA to help their agents, with no limits on what the agents are allowed to get as help because their bosses trust it's always in the service of the nation, will quite "happily" write that essay… and almost certainly get leaked eventually — 10 years, 50 years, doesn't matter — at which point someone who isn't so trustworthy will ask for a doomsday plan and it will be equally "happy" to write that too.

Hopefully the agencies like the FSB and CIA aren't quite as bad as I fear, but secrecy and what little has become public makes it impossible to trust them.

Re:Evolution of AI and Amara's Law by nerpderp82 on Monday January 22, 2024 @06:23PM (#39097023)
AGI doesn't matter, because the world will have been turned upside down multiple times before it even arrives. AI due to its extreme breadth is already smarter than any individual human across all of AIs competencies.
Re:Evolution of AI and Amara's Law by Terr_ on Monday January 22, 2024 @08:01PM (#39097943)
I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting.
Re:Evolution of AI and Amara's Law by Terr_ on Monday January 22, 2024 @08:02PM (#39097953)
I feel that even in just the phrasing of the question, "wants" is doing an awful lot of heavy lifting. It's almost tied in a kind of "No True AGI" framing, where if it doesn't want something it's not really AGI, etc.