My position is that it is either right around the corner or that it is 50 years and several major tech breakthroughs away. The reason is that there is now so much money poured into this space that you are essentially doing an exhaustive search around the point already reached. As long as that results in more breakthroughs more money will be available. If that stops working for a while then the money dries up and we're in a period of quiet again. So keep an eye on the rate of improvements and as long as no single year passes without a breakthrough it could happen any time.

If we're just 50 years away from an existential risk, that does strike me as an urgent concern and a narrow timeline.

It is. Show me a single individual with a time horizon that exceeds their own life or that of their children. That means you have to be early 20's or so today (or late teens) and that you are aware enough and capable enough to act on something that is not an immediate concern for decades. I don't see that happening, even if a handful of such individuals exist it won't be enough to make a difference, and that's before we get into cultural differences which undo your efforts because others are going full steam ahead anyway and probably in much larger numbers.

I mean we are 50 years away from a lot of existential risk and the average guy does not give a fuck.

People told me the same thing about self driving cars 10 years ago.

Waymo works pretty well today in the Bay Area. Self driving cars are here. They’re just not evenly distributed.

On days with perfect weather, with heavy reliance on extremely detailed pre-computed maps, with constant internet access, on paved roads, in scenarios it has encountered before. Getting from where we are to what is promised is not trivial at all.

AGI is a needed part of self driving in unsupervised environments.

We put the cart before the horse. Now in semi controlled environments self driving cars do pretty well.

But in an unsupervised environment I know if a ball is rolling towards the road a child may be following it. What's interesting is you feed GPT a picture of a ball going toward a road it can make guesses that it presents risks because a person may be following it.