> Seems like being directionally right early that AI would be a Big Deal and scary should count for something?

If I scream that a vicious beast is going to destroy the world, I don't get credit for being 'directionally right' if a squirrel eats a hazelnut. AI being a big deal is a long, long way from the full beliefs Yudkowksy promotes, and there are many people who predicted AI's significance who didn't also believe it would be an extinction event.

This doesn’t track though. Just recently AI became “powerful enough” to warrant export controls from the US and maybe soon China.

It’s more he screamed that bacteria is gonna do evolution faster than any before it and turn into a beast that destroys the world, and now that bacteria has turned into a squirrel so far. He’s been very right so far and if that squirrel keeps sizing up you’d want to give him credit.

  Just recently AI became “powerful enough” to warrant export controls from the US and maybe soon China.

What’s the term I’m looking for here? This is one of the most absurd statements I’ve ever seen but it feels like a pattern I’ve forgotten from books or something.

Someone makes a made up claim, second party acts on made up claim, third party takes second party’s actions a proof of claim.

Reminds me of the “they’re eating pets” thing a bit.

Do you mind taking your comment a bit further to a conclusion so I can better understand your perspective?

Like are you saying that AI is still kinda not a powerful tool and unlikely to become geopolitically relevant etc? I don’t know how to relate this to culture war media frenzies which is rather stay out of.

It's not a particularly novel claim since at least Terminator (and even moreso Terminator 2) made it EXTREMELY mainstream.

It was a topic in less-mainstream sci-fi well before that. And some more mainstream stuff like Star Trek TOS.

Frankly it seems more common than not in the last 40 years. I don't really remember a big wave of claims "Terminator is silly, no sort of AI could ever be malevolent!"

One thing I find weird is that despite the stock photo of the Terminator's shiny endoskeleton holding a gun being used to illustrate so many AI news stories, now we have ChatGPT people keep saying "how can AI possibly hurt us?"

We've even had the dichotomy of some Doctor Who episodes where the self-driving car crashes itself to kill the occupant to silence them, against repeated real-life news stories about self-driving cars killing people, and yet the connection isn't getting made that software controls hardware.

> Seems like being directionally right early that AI would be a Big Deal and scary should count for something?

Only if it was an uncommon prediction; otherwise, it’s just evidence of common sense.

It's common sense now. It hasn't always been common sense that AI alignment is an important problem.

It depends on what you mean by alignment. If you mean the sense it's used now, meaning bias in models, security and safety for users and so on, then that's a way more mundane version of alignment than what Yudkowsky and ilk promoted. They are talking about a superintelligent being possibly destroying humanity / the earth. And it wasn't that hard to predict. We have many examples of more technological / "advanced" people ruling or extinguishing weaker people. So a superintelligent AI being a threat is not a huge leap.

IMO they have been consistently too aggressive on timelines. When GPT-4 came out, Yudkowsky said it might be conscious. I think he has written interesting stuff but let's be real

> When GPT-4 came out, Yudkowsky said it might be conscious. I think he has written interesting stuff but let's be real

I don't expect GPT-4 to be conscious, it would be quite remarkable if we've stumbled upon whatever evolution did that made us conscious.

But how would you even tell, given we don't know the mechanism or physical process that un-thinking and un-planning natural selection happened to stumble upon to give it to us?

Sure, we can't tell, but that's a really bad argument to make the claim that it _is_ conscious. I just have a really hard time believing that a transformer can be conscious, but yes I can't prove it.

Agreed. I push back equally against all who are confident, regardless of which way.

If we do a survey of stories about AI from the last 50 years, it seems like it's a worry everyone knows about, and that people generally take seriously in proportion to how much they think an AI will actually exist.

Why? Science fiction talked about technology’s behavior and human expectations of it even way before computers became mechanical.

Stories about golems going rogue and genies requiring careful wishing aren’t modern era inventions.

It's a "big deal" to who? People in a bubble on HN? For sure. But the rest of the masses? Do they think it's a big deal, do they care? Really they shouldn't. AI has very little to show for benefits at this point for the everyday, average human being.