Your hypotheticals would be nice.

I think people are just failing to see the progress as exponential and to extrapolate that way. I see a lot of 'pooh pooh' claims that make this assumption that because the current state is flawed or doesn't measure up that somehow justifies the belief that they can rest easy and have nothing to worry about. I contrast that with my own experience of I go to bed and wake up the next day and check what new tools, models and papers have been released and usually have one "hot damn!" moment per day.

We’re on the zero downside, only upside technology curve to utopia ?

>We’re on the zero downside, only upside technology curve to utopia ?

Not how technology works in my experience. My mental model of technology is that it acts as a multiplier on life's peaks and valleys, so the highs get higher and the lows get lower. To my mind that means AGI will bring the ultimate of highs along with the ultimate of lows. Doesn't sound like utopia to me. Sounds like a good time, until it's not.