I'm pretty sure we're far away from real AGI too, but the issue that worries me most is if these models advance along current trends suggest, in a few years we could be facing a situation where large swathes of the population are suddenly redundant, and labor loses all ability to stand up to capital. That's a kind of political situation that could easily escalate into a total collapse of society into a totalitarian nightmare.