I think they mean we don’t have a deathstar. We can make the atoms here shitty. We can’t annihilate them or scatter them out of their own gravity well.

Life in theory could restart with our weapons, but what’s the point with the other scenario.

Being scared about novel AI weapons sounds like the people who were scared that turning on the LHC would create a black hole that would suck in the Earth.

The most important difference is that this time it's the people who know the most about the topic who are concerned, and the people who know the least about it who are blasé. (Also, Yann LeCun.) With the LHC it was the other way around.

If you truly believe this, I am begging you to touch some grass, preferably very very far away from Silicon Valley. It’s nice out here.

My demographic claim? It's obviously correct, as anyone with the most casual acquaintance with the question knows. The worried people, like Geoffrey Hinton, Bill Gates, and Dario Amodei, might be wrong, but they certainly aren't ignorant outsiders.

How would touching grass help? Do you think the grass is better informed about AI risk than you are?

...that's plausible, actually.

Probably a site called "Hacker News" is not the smartest place for you to be implicitly asserting that thinking about things is bad, or that those nerds who do it are out-of-touch losers.

Bill Gates is too busy trying to save his reputation from his Epstein association to be believed, he's a billionaire concern troll and Amodei is far more worried about his stock price than he is about humanity. Those two are discounted. That doesn't mean we don't have a problem, but they are just hitching their wagons to horse that happens to be running in a direction that is temporarily useful to them.

That's plausible, although I don't think it's actually correct. But there's a pretty extensive set of AI luminaries who are concerned who can't be dismissed so easily.

Spreading fear of an awesome power whose true danger is unproven is Religion 101.

Also scamming 101 for that matter, you want your mark to feel an immediate threat of overwhelmingly negative consequences so they'll buy whatever you're selling to prevent it. Silicon Valley directly makes tens of billions of USD literally serving scams to the public already, so I think it's fair to take most claims of doom with a pinch of salt.

It's also probably the correct approach to take with an awesome power whose true danger is unproven.

There are as many 'luminaries' on LeCun's side as there are on Hinton's. Andrew Ng, Pedro Domingos, Schmidhuber, and the list goes on.

Any time it seems to you that there are more people on the side of an issue that you happen to be on yourself, a good rule of thumb is to assume confirmation bias.

The list doesn't go on. I'll give you Ng and Schmidhuber, but your list pretty much ends there. Domingos isn't even a neural networks guy, and he's retired.

I agree that confirmation bias is a good working assumption in cases like this, but in this case we can conclusively reject that assumption. On the "worry about AI risk" side, we have the 33,705 signatories of https://futureoflife.org/open-letter/pause-giant-ai-experime..., starting with Yoshua Bengio, Emad Mostaque, and John Hopfield, one of the few people to have actually won a Nobel for neural-networks work. And the 1386 signatories of https://www.pacingthefrontier.com/, starting with John Schulman, Jakub Pachocki, Jared Kaplan, Shengjia Zhao, Shane Legg, Ilya Sutskever, Mark Chen, Jasjeet Sekhon, Dario Amodei, Jack Clark, Anca Dragan, Wojciech Zaremba, Dawn Song, Chris Olah, Laura Weidinger, Benjamin Mann, Stephanie Chan, Julian Schrittwieser, Summer Yue, and Jan Leike.

They might be wrong, but the demographics of the concern are pretty clear. AI existential risk is not a fringe concern raised by some group of Luddites due to their ignorance about the science, as in the black-hole risk of the LHC. It's primarily AI researchers, and especially the AI researchers who are working most closely with the most successful technologies.

By contrast, to come up with three more luminaries on LeCun's "side", you had to trot out a retired machine-learning professor who used to do research on Markov chains and Bayesian inference.

It's very disappointing to hear BS like this coming from a very intelligent crowd such as HN. How many people in the US do you think know who Epstein was, let alone what is contained in his file? How many in the world? If Bill Gates is worried about his reputation, Microsoft has sealed the deal long time ago.

As for Dario, of course he's worried about the stock price. I would if I were in his shoes. That's literally part of a CEO's job. He is also one who has consistently raised the issue with AI safety. He was OpenAI's head of AI safety after all. Anthropic is considered security risk by the US. Is that "temporary useful"?

Do you have an argument to make or only snide condescension?

> I think they mean we don’t have a deathstar. We can make the atoms here shitty. We can’t annihilate them or scatter them out of their own gravity well.

What a silly distinction to make.

> probably destroy almost anyone who survived the initial blast and fallout.

The difference between killing almost everyone, and killing everyone, is similar to the difference between almost getting pregnant and getting pregnant. A hundred thousand years later, a world where almost everyone had been killed would be teeming with humans.

> Can you explain how exactly an AI would develop a novel planet-killing weapon

I already answered that question in my previous comment, and the answer is that I am not going to explain how to develop weapons, but the larger point here is that we already can't explain how exactly AIs do innumerable things we see them doing every day, which is why we're having this conversation in the first place.

> I seriously don’t understand this argument

That is a point that I think everyone can agree on: you do not.

>Can you explain how exactly an AI would develop a novel planet-killing weapon without anyone noticing and pulling the plug? Like I seriously don’t understand this argument.

That's the danger with creating a super intelligence, that you won't understand it. My dog is pretty smart, has good instincts and I'm quite confident he can survive in nature all by himself. But if a human decided to harm him, he doesn't stand a chance.

To be fair, he could probably fuck the human up pretty good if he had a mind to, too. Which may be the most likely reason that a human might decide to harm him: if he bites a couple of children, for example, or if a burglar wants to rob your house.

> That's the danger with creating a super intelligence, that you won't understand it

This argument is sophistry because it assumes the conclusion in its premise. It works backwards from extinction by proposing a hypothetical computer program that definitionally cannot be understood, predicted or controlled.

A substantive argument would propose reasoning that explains why we should believe such a program could be created.

No, that is not how reasoning works. It is perfectly valid reasoning to begin by positing entities that we may not be possible to create, such as supersonic airplanes, coherent light, quantum computers, rockets that can reach escape velocity, black holes, or infinitely long lines that are perfectly straight and infinitely thin. Indeed, you might say that the ability to reason about the consequences of premises that we do not know to be true is the thing that distinguishes reasoning, as such, from simple parroting.

It could very well be the case that creating superintelligence is impossible. We won't know for sure unless it is achieved. Nevertheless, we can use reasoning from the hypothesis that it is possible to predict some of the effects that would ensue were it to be achieved.

This reasoning, however valid it may be, is of course of much less interest should it turn out that we are reasoning about an impossible situation. Many arguments have been suggested on both sides of the issue, but the arguments for impossibility are extremely weak; most of them are conclusively dispatched in https://arxiv.org/pdf/1703.10987.

AI could invent something like social media.

Seems like bioweapons (assuming "planet killing" means killing all the people, not literally blowing up Earth) are one way. Convincing the creator it would achieve their aim and could be contained when in fact it could not.