i think we need to "survive AGI"
a powerful international coalition similar to NATO could exclude the possibility of a rogue nation or entity developing AGI. its a very expensive and arduous process for a small group -- you cant do it in your basement. the best way to think about it is that all we have to do is not do it. its easy. if an asteroid was about to hit earth, there might be literally nothing we could do about it despite the combined effort of every human. this is way easier. i think its really ironic that the worst disaster that might ever happen could also be the disaster that was the easiest to avoid.
do you think china/any totalitarian govt would follow suit with that regulation? if so, why?
> a powerful international coalition similar to NATO could exclude the possibility of a rogue nation or entity developing AGI.
how?
how? the same way that powerful international coalitions do anything else... with overwhelming economic and military power.
(The Fermi paradox is also the kind of thing discussed on LessWrong.)
We're racing into a fundamentally deep and irreversible societal shift, at least the same order of magnitude as the agricultural or industrial revolution. Maybe even many orders of magnitude deeper. Society will change so profoundly, it will be at least as unrecognizable as our lives would look to the average person from the Bronze age. There's absolutely no reason to assume this will be a good change. If it's not something I personally will have to live with, my descendants most certainly will.
I'll admit, I also draw a blank when I try to imagine what the consequences of all this will be, but it's a blank as in "staring into a pitch black room and having no idea what's in it" - not ignoring the darkness altogether. Mass psychosis is a good term for this, I think.
The collective blindspot failing to understand that there's NOTHING that says we're gonna 'make it'.
There's no divine being out there watching out for us. This isn't a fucking fairy tale, you can't assume that things will always 'work out'. Obviously they've always worked out until now because we're able to have this conversation, but that does NOT mean that things will work out indefinitely into the future.
Baseless conjecture: I think we are biased towards irrational optimism because it's an adaptive trait. Thinking everything will work out is better than not, because it means you're more likely to attempt escaping a predator or whatever despite a minuscule chance of success (which is better than not trying at all). It's another entry into the list of instincts we've inherited from our ancestors which bite us in the ass today (like being omnivorous, liking sweets, tribalism, urge to reproduce, etc).
You seem like you've given this a bunch of thought, and I wanna chat more about this and pick your brain about a few things. Have you ever thought about whether this intersects with the Fermi paradox somehow?
Drop me a line here: l7byzw6ao at mozmail dot com