And it is very clear that the AI companies do want you to be scared of the former, because it makes their tech seem more impressive and distracts from its very real flaws.
Stupidity has of course always existed and you can't regulate it away, but modern tech and the business interests behind it seem to me to be making it worse and potentially more destructive.
I'm not talking about just the latest hype here, it has been a trend for more than a decade now: "don't think for yourself, just consult the magic oracle in your pocket that surely knows best and isn't purposely designed to manipulate you into buying more crap and subscribing to the worldview of a small elite"
But like with the AI apocalypse, or any conspiracy theory, the really scary thing is that there doesn't have to be some kind of intelligent plan behind it, something that you could understand or fight against, or maybe join in because you think they have the right goals. Rather, it's just mindless forces of the market and social dynamics.
I really think it's all going to collapse within another 15 years or so, and can only hope that civilization will recover and perhaps learn from the mistakes made.
"They're the same picture".
Imagine someone who wants to rule the world, just asks an AI for such a plan. AI doesn't need to want anything itself, just be given free rein. I'm sure Boris Johnson would have taken this option if it had existed, though I doubt it would have made any difference to the character flaws which were his ultimate downfall.
Even the "paperclip maximiser" scenario is just someone in the business of making paperclips asking an AI for help, and who doesn't look too closely until it's too late.
Myself, I'm… relatively optimistic, in that I think these systems are likely to be fragile to distribution shifts they themselves create, in ways that mean the rest of us can likely stop them from being existential threats if they start down a dangerous path. (This requires that we never figure out how to make them learn from as few examples as humans need, which may be wishful thinking, but for now seems to be an acceptable guess).
For some reason this reminded me of the old joke about the engineer, the physicist, and the mathematician's differing approaches to noticing a fire; I hope humanity doesn't take the mathematician's solution: https://jcdverha.home.xs4all.nl/scijokes/6_2.html
Irony in the "Greek tragedy" sense (but who is εἴρων and who is ἀλαζών?)
So many online arguments about AI safety, where someone says "it's fine: just don't let it out of the box", and here it is, accessible on the internet — and for some, even this isn't enough: they want the weights; and for some, even the weights aren't enough, they want the training data.
So many arguments where someone says "it's fine: just switch it off if it goes wrong", and not only is the switch out of reach to those affected, most people aren't even checking if it's going wrong in the first place.
The labs themselves ring the alarm bells, put disclaimers on the start page of their own products, call for binding regulations, and there's always someone dismissing this as a "5D chess" marketing move.
So many where someone says "it's just speech, what can it do?", etc. etc.
I'm glad the AI labs themselves have so many signatories to the "let's pause for a bit" open letter. Cool though the tech is, it's definitely acting outside the expectations many have for it, and a simple dichotomy of "better/worse" isn't sufficient to describe the ways in which it is outside expectations given it can be both at the same time.