Re:Evolution of AI and Amara's Law
by visarga on Monday January 22, 2024 @05:17PM
(#39096286)
I think I read this kind of discussions too many times, imagine how the training set of GPT-5 will look like? all sorts of theories of "whether AGI wants to help or kill the human race" spread over the internet and analyzed to death. It is going to know this topic in-and-out. Will be able to write a masterful dissertation on the topic.
[ Parent
| on HN ]
Yes, and that's good because GPT-5 is going to be coming from a team that cares about alignment and which has values only a little different from mine. They can still mess up: https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt
OTOH, an LLM trained by the FSB or CIA to help their agents, with no limits on what the agents are allowed to get as help because their bosses trust it's always in the service of the nation, will quite "happily" write that essay… and almost certainly get leaked eventually — 10 years, 50 years, doesn't matter — at which point someone who isn't so trustworthy will ask for a doomsday plan and it will be equally "happy" to write that too.
Hopefully the agencies like the FSB and CIA aren't quite as bad as I fear, but secrecy and what little has become public makes it impossible to trust them.