News for hackers. Stuff that matters.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @05:41AM (#38111070)
There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad.

Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors.

Unaligned ASI though, could actually make us extinct.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by bryanrasmussen on Thursday November 02, 2023 @05:57AM (#38111181)
it said end of civilization?

>Horrible, horrible stuff, but not extinction.

anyway climate change drop single percentage directly caused, but that kind of thing seems like it would have LOTS of side effects.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by kylebyte on Thursday November 02, 2023 @06:11AM (#38111287)
I'm pretty sure the worst case of your nuclear war scenario is actually the worst case of the climate change one.

Chunks of the world suddenly becoming unlivable and resources getting more scarce sounds like a recipe for escalation into war to me.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by ethanbond on Thursday November 02, 2023 @07:19AM (#38111833)
The nuclear war scenario, according to all known information about nuclear war policies, is not “chunks of the world becoming unlivable.”

The principals in a nuclear conflict do not appear to even have a method to launch just a few nukes in response to a nuclear attacks: they will launch thousands of warheads at hundreds of cities.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by ImPostingOnHN on Thursday November 02, 2023 @09:38AM (#38113339)
I suppose thousands of warheads could be launched at hundreds of cities, and that could make chunks of the world uninhabitable and kill millions

climate change will happen, and it will do that, left unchecked

Re:Yann LeCun: AI one-percenters seizing power forever is re... by ethanbond on Thursday November 02, 2023 @10:53AM (#38114302)
I'm not arguing whatsoever against action on climate change, I'm just articulating actually how bad a nuclear exchange would be. It's far, far, far worse than most people imagine because they (understandably) couldn't fathom how monstrous the actual war plans were (and are, as far as anyone knows).

Daniel Ellsberg, the top nuclear war planner at RAND during the Cold War, claims that the Joint Chiefs gave an estimate to Kennedy in 1961 that they'd expect 600,000,000 deaths from the US's war plan alone.

That's 600 million people:

1. At 1960s levels of urban populations (especially in China, things have changed quite a lot -- and yes the plan was to attack China... every moderately large city in China, in fact!)

2. Using 1960s nuclear weapons

3. Not including deaths from the Russian response (and now Chinese), at the time estimated to be 50 - 90 million Americans

4. Before the global nuclear winter

Re:Yann LeCun: AI one-percenters seizing power forever is re... by OkayPhysicist on Thursday November 02, 2023 @06:31PM (#38121267)
That's not extinction, that's not even close. Maybe, just maybe, it'd be the end of both western and eastern civilization, but it's nowhere near wiping out all life on Earth.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by ethanbond on Thursday November 02, 2023 @08:51PM (#38122842)
I don’t think anyone suggested it was
Re:Yann LeCun: AI one-percenters seizing power forever is re... by kylebyte on Friday November 03, 2023 @03:57PM (#38134168)
I was unclear, I meant chunks of the world becoming unlivable from climate change could easily be the catalyst for a nuclear war. So the worst case for both is the same in my mind.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by shafyy on Thursday November 02, 2023 @06:24AM (#38111398)
What is ASI and how could it make humans extinct?
Re:Yann LeCun: AI one-percenters seizing power forever is re... by concordDance on Thursday November 02, 2023 @06:34AM (#38111488)
Artifical Super Intelligence. Could make humans extinct just like humans could make chimpanzees extinct. Using superior intellect to better leverage available resources than the things it/they are competing with until it out expands/outcompetes them.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by TeMPOraL on Thursday November 02, 2023 @06:58AM (#38111675)
Note that ways ASI can make us extinct includes all the ways we could do it to ourselves.

One of the possible scenario is tricking humans into starting WWIII, perhaps entirely by accident. Another is that even being benignly applied to optimize economic activity in broad terms might have the AI strengthen the very failure modes that accelerate climate change.

Point being, all the current global risks won't disappear overnight with the rise of a powerful AI. The AI might affect them for the better, or for worse, but the definition of ASI/AGI is that whichever way it will affect them (or anything in general), we won't have the means to stop it.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by shafyy on Thursday November 02, 2023 @09:58AM (#38113589)
> One of the possible scenario is tricking humans into starting WWIII

This sounds more like a plot of a third-grade movie than something that could happen in reality.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by NoGravitas on Thursday November 02, 2023 @12:21PM (#38115852)
It's the plot of the Terminator franchise (more or less; in the Terminator franchise, the AGI was given permission to launch on its own authority, as it was believed to be aligned with US military interests. But before that, it tricked humans into building the automated factory infrastructure that it would later use to produce Terminators and Hunter-Killers.).
Re:Yann LeCun: AI one-percenters seizing power forever is re... by ben_w on Thursday November 02, 2023 @01:28PM (#38117046)
Both the USA and the USSR have come close due to GOFAI early warning systems:

https://en.wikipedia.org/wiki/Thule_Site_J#RCA_operations

https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...

Post-collapse Russia also came close one time because someone lost the paperwork: https://en.wikipedia.org/wiki/Norwegian_rocket_incident

Re:Yann LeCun: AI one-percenters seizing power forever is re... by pixl97 on Thursday November 02, 2023 @02:40PM (#38118094)
If I snached someone out of 1850 and presented them the world in which we live the entire premise of our existence would seem to be nonsensical fiction to them, and yet it would be real.

And using movie plots as a basis generally doesn't directly work as fiction has to make sense to sell whereas reality has no requirement of making sense to people. The reality we live in this day is mostly driven by people for people (though many would say by corporations for corporations) and therefore things still make sense to us people. When and if an intellect matches or exceeds that of humans it can easily imagine situations where 'the algorithm' does things humans don't comprehend, but because we make more (whatever) we keep giving it more power to do its job in an autonomous fashion. It is not difficult to end up in a situation where you have a system that is not well understood by anyone and you end up cargo culting it to hope it continues working into the future.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by TeMPOraL on Friday November 03, 2023 @09:01AM (#38128050)
> 'the algorithm' does things humans don't comprehend, but because we make more (whatever) we keep giving it more power to do its job in an autonomous fashion. It is not difficult to end up in a situation where you have a system that is not well understood by anyone and you end up cargo culting it to hope it continues working into the future.

In fact, this is a perfect description of what the market economy itself is right now.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by devnonymous on Thursday November 02, 2023 @06:38AM (#38111522)
Climate change has compounding effects (eg: water scarcity, leading to political conflict, leading to displacement, leading to pandemics ...and that's just one vector). Real world is real real messy. COViD should have taught people that. Unfortunately, it didn't.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @09:06AM (#38112974)
I am not saying it's not a huge problem. Hundreds of millions dying would probably make it the worst thing that ever happened.

I am just saying it's not an extinction risk.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by devnonymous on Thursday November 02, 2023 @09:18AM (#38113112)
...tbh, imo, the impact of this is already visible in the world (eg: it can be argued that the conflict in Ukraine is directly linked to climate change) whereas ASI is still fantasy that exists only in the minds of the AI 1% , to use the terminology of the OP.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by NoGravitas on Thursday November 02, 2023 @12:32PM (#38116055)
I believe you are correct on both points. I believe the first major war to be caused substantially by climate change was the Syrian Civil War, followed by the Yemeni Civil War.

The main AI threats for the foreseeable future are its energy usage contributing to climate change, and the use of generative AI to produce and enhance political unrest, as well as its general use in accelerating other political and economic crises. AGI isn't even on the radar, much less ASI.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by itsoktocry on Thursday November 02, 2023 @09:15AM (#38113068)
>Climate change has compounding effects

Why is the default to assume that every change will be negative?

Re:Yann LeCun: AI one-percenters seizing power forever is re... by devnonymous on Thursday November 02, 2023 @09:19AM (#38113124)
Perhaps because we aren't seeing evidence to the contrary?
Re:Yann LeCun: AI one-percenters seizing power forever is re... by IX-103 on Thursday November 02, 2023 @12:14PM (#38115727)
Because changes in climate mean that the existing infrastructure (in the form of cities, farmland, etc) that are optimized for the current conditions are no longer optimal. Very few cities are designed for conditions below sea level or in a desert, but if the city already exists and the desert and oceans comes to where they are then bad things will happen.

I mean you're right that conditions in some places might get better, but that doesn't help if people have to build the infrastructure there to support a modern settlement in order to take advantage of it. When people talk about the cost of climate change, they're taking about the costs of building or adapting infrastructure.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by ijk on Thursday November 02, 2023 @02:59PM (#38118396)
Not to mention that most species evolved to fit particular environments and don't do well when you rapidly alter those environments far outside the parameters they tend to live in.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by M95D on Thursday November 02, 2023 @06:39AM (#38111526)
> He's pretty sure that human civilization will be extinct this century.

> There is no scenario where climate change leads to human extinction.

He was talking about the human CIVILIZATION...

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @07:16AM (#38111806)
You are right, but "human civilization will be extinct" is an awkward phrase.

Human extinction via climate change is off the table.

The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's the average, it will be much worse in some regions).

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @07:18AM (#38111827)
Sorry, I meant IPCC (International Panel on Climate Change).

I don't think IUPAC (International Union of Pure and Applied Chemistry) has come out with views on impact of global warming :)

Re:Yann LeCun: AI one-percenters seizing power forever is re... by selimthegrim on Thursday November 02, 2023 @01:02PM (#38116597)
They can’t be doing any worse than APS
Re:Yann LeCun: AI one-percenters seizing power forever is re... by mangosteenjuice on Thursday November 02, 2023 @07:18AM (#38111829)
Right, the collapse of our current civilization is inevitable. That doesn't mean it won't be succeeded by more advanced ones in the future
Re:Yann LeCun: AI one-percenters seizing power forever is re... by M95D on Friday November 03, 2023 @03:41AM (#38125658)
Once it collapses, it won't rise again to even an industrial level civilization. Iron ore, copper, coal are no longer easy-accessible. Can't have a new industrial age without those.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by mannykannot on Thursday November 02, 2023 @08:07AM (#38112360)
> If it means we'll go back to being hunter gatherers it's ridiculous.

Indeed, but even going back to the 19th century would have dire consequences, given our current global dependence on the industrial fixing of nitrogen.

If civilization were to collapse (and I don't think it will, other than from a near-extinction event), I doubt it would be like going back to any earlier time.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @09:09AM (#38113004)
Oh, for sure. I am just saying it's not an extinction risk, not that it is not a huge and pressing problem.

I would still bet against going back to 19th century. Worst case scenario for me is hundreds of millions dying and a big regression in global standards of living, but keeping our level of civilization. Which is horrible, it would be the worst thing that ever happened.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by wcoenen on Thursday November 02, 2023 @07:38AM (#38112007)
Climate change and nuclear war are not orthogonal risks.

Climate change leads to conflict. For example, the Syria drought of 2006-2010.

More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @09:13AM (#38113056)
I don't disagree with that, but then the risk is nuclear war. It could be triggered by global warming conflicts or by something else.

Still, like I said somewhere else, even all out nuclear war is unlikely to lead to human extinction. It could push us back to the neolithic in some scenarios, but even then there is some disagreement about what would happen.

Of course, even in the most optimistic scenario it would be really bad and we should do everything we can to avoid it - that goes without saying.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by ImPostingOnHN on Thursday November 02, 2023 @09:35AM (#38113321)
Then, actually, the risk is the root cause: climate change, not nuclear war. The global war could be conventional, too. Or chemical, or biological, or genetically engineered. No matter the tool used, the risk is why they would use it vs. not using it now.

In any case, besides addressing the root cause vs. proximal cause, you couldn't even address the proximal cause anyways: it's more likely that the world could do something about climate change than about war in general.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @11:05AM (#38114510)
Well, it's a matter of opinion which one is the root cause.

My take is that if you remove nuclear/biological war out of the picture somehow, climate change is not an existential risk. If you remove the latter then the former is still an existential risk (and there are unfortunately a lot of other possible sources of geopolitical instability). So the fundamental source of risk if the former. But it's a matter of opinion.

Conventional or chemical warfare, even on a global scale, are definitely not existential risks though. And like I said, probably not even nuclear. Biological... that I could see leading to extinction.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by lettergram on Thursday November 02, 2023 @09:19AM (#38113128)
India and Pakistan didn’t nuke each other last time(s)… even though they had nuclear weapons.

The assumption we’d need to use nukes is insane. For the same cost (and energy requirements) for a set of nukes we can filter salt from ocean water or collect rain.

I agree famine and drought can cause conflict. But we are no where need that. If you read the climate change predictions (from the UN), they actually suspect more rain (and flooding) in many regions.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by wcoenen on Thursday November 02, 2023 @10:54AM (#38114321)
> India and Pakistan didn’t nuke each other last time(s)… even though they had nuclear weapons.

I was referring to the Indo-Pakistani war of 1947-1948 where the conflict focused on water rights. Nuclear weapons did not enter the picture until much later.

Earlier this year, those old tensions about water rights resurfaced:

https://www.usip.org/publications/2023/02/india-and-pakistan...

> For the same cost (and energy requirements) for a set of nukes we can filter salt from ocean water or collect rain.

The fight would be over the water flowing through the Indus, which is orders of magnitudes more than all Indian and Pakistani desalination projects combined.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by api on Thursday November 02, 2023 @07:40AM (#38112044)
> There is no scenario where climate change leads to human extinction

https://en.m.wikipedia.org/wiki/Clathrate_gun_hypothesis

Re:Yann LeCun: AI one-percenters seizing power forever is re... by toth on Thursday November 02, 2023 @09:05AM (#38112953)
Interesting, I have some vague recollections off hearing about this.

But according to your link, IPCC says it's not a plausible scenario in the medium term. And I'd say that even 8 degrees of warming wouldn't be enough for extinction or end of human civilization. But there it could be double digit percentage of human population dying.

Re:Yann LeCun: AI one-percenters seizing power forever is re... by CuriouslyC on Thursday November 02, 2023 @09:00AM (#38112890)
There is a scenario. The climate has been buffering global warming, and as its ability to do that fails, the rate of temperature increase accelerates along with many extreme climate events. This rapid acceleration and series of catastrophes causes a combination of mass migrations along with unprecedented crop failures. First world countries are destabilized by sky high food prices and the mass influx of migrants, leading to a big uptick in wars and revolutions. Eventually mass starvation pushes us into wars for food that quickly spiral into mutual annihilation.
Re:Yann LeCun: AI one-percenters seizing power forever is re... by germandiago on Thursday November 02, 2023 @10:20AM (#38113866)
I always hear all this. Russia has thr biggest surface in the world for a country. A big part of it is frozen... Is that... bad? To give just an example and from my extremely limited knowledge on the topic.

But it is a topic that is extremely biased towards some interests anyways

Re:Yann LeCun: AI one-percenters seizing power forever is re... by mym1990 on Thursday November 02, 2023 @10:26AM (#38113946)
I don’t think they are arguing that this can’t happen, but that it is unlikely to kill every single human. I mean climate spans very very long time frames, so certainly if we can’t find a way to hang on by a thread and keep reproducing, it will spell the end.