Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization | Lex Fridman Podcast #368
I'm convinced the threat is real, but have no idea what the timeline is. I hope, like most things, we'll skate by, and just stop calling it AI once it happens, and treat it like any other tool. I strongly doubt that is true.
I suspect what will actually happen is that peak oil will catch us off guard, and we won't have the spare power available to train GPT7, and that will avert the singularity.
I see the danger, let me give an analogy.
What if, according to the laws of physics, it were possible to make a thermonuclear weapon out of beach sand using a microwave oven.
That's something so absurd that we'd never figure it out, but AGI could. That scale of dangerously destabilizing knowledge could show up at any time from a superintelligent AGI.
Its bad enough that nation-states have the resources to make civilization ending weapons. I think AGI could super-empower those with access to it.