Good my ADHD makes me write HN comments on LLM threads instead of working on the latter, because I'm not sure I could stop myself otherwise. But this also means I can very much imagine only thinking about the cool parts, and not considering the dangerous ones - because that's how I feel about 90% of the time when thinking about current SOTA models. It's just too damn exciting.
(Hell, it's the first thing in tech in the last decade, that I find exciting.)
--
[0] - Whatever one may think of them, I find it sadly ironic how Eliezer and his disciples were considered pseudo-scientific doomsday AI cultists a decade ago, back when the whole thing was only a far-future concern, how they're still considered pseudo-scientific doomsday AI cultists today, after AI field made a series of rapid jumps, landing us exactly in the scope of things they were harping about all that time - and how I bet they'll still be considered pseudo-scientific doomsday AI cultists after someone unleashes an unaligned AGI and ends us.
I know it's technically crazy sounding, but for all we know, Earth is the only place with biological life, I think it's better to flip the coin somewhere else if that's what you want to do. Don't screw up here.
Yes, maybe the AI will come for Earth, but I feel like there's less chance of shit going really bad with some distance.
The doomsday AI cultists didn't come up with any of these ideas. The Matrix was a hugely popular movie in 1999.
chatGPT doesn't make the person who has believed since 1999 that AI is going to turn us all into Duracell batteries more correct. It is the same basic thought with with paper clips replacing batteries.
That's the worst part of it.
Is the current generation of LLMs dangerous? Probably not. Will the next generation of LLMs be dangerous? Maybe not, if we're lucky. The one after that? Nobody knows.
But projects like this don't so much rely on there being no danger, but rather, the question seems to not even have been considered. And that's crazy, because the assumption that things are harmless is getting more and more shaky the more capable these models become.