But a lot of serious people worry about the alignment problem when applied to future, actually "intelligent" (in the sense of having powerful planning ability), AIs.
We don't know when they'll be possible, but we do know that we don't currently know how to train them in a manner where they'll behave as we like when far outside the training domain (we cant even clearly define what we'd like!).
This is actually a very complex topic and I wouldn't be able to do it justice in a hackernews comment.
See https://en.wikipedia.org/wiki/No_true_Scotsman for a rebuttal to your dismissal.
Maybe by the fact that big LLMs have sophisticated behaviors that no one understands and/or can explain yet?
In 1962, John F. Kennedy famously said, "Those who make peaceful revolution impossible will make violent revolution inevitable." But AI can make violent revolution impossible as well, by tracking and surveilling and intervening. The fewer people are needed to maintain such a system, the easier the end of democracy will be. It's not going to happen today, or tomorrow, but that's what I'm betting my money on for "end of society as we know it" scenarios.
You can scratch that one off the list. This is pure pseudo-intellectual fantasy. This notion isn't backed by anything and there are good reasons to disregard it, to put it mildly. It is fair to say that this concern isn't even wrong.
The concerns over "AI" have been and continue to be of a practical nature, namely, that of increasingly sophisticated automation. If we frame things in terms of automation, we can have a more productive discussion that stands of chance of contributing to good legislation and regulation instead of mystification and obscurantism.