The problem isn't AGI becomes the oracle central planner. The problem is AGI becomes the central planner, the government and everybody else who currently has a job.

I think there won't be just one AGI, no central planner. LLM abilities leak, other models can catch up in a few months.

The problem with all such criticisms is that there is an implicit assumption that humans can be trusted.

I know human limitations. I don’t know AGI limtations.

Most humans can not lie all the time. Their true intentions do come out from time to time.

AGI might not have that problem - AGI might hide its true intentions for hundreds of years.

Have you… met humans?

I don't think the concern is that an AGI would become a central planner, but that an AGI would be so much better than human investors that the entire VC class would be outclassed, and that the free market would shift towards using AGI to make investment/capital allocation decisions. Which, of course, runs the risk of turning the whole system into a paperclip optimizing machine that consumes the planet in pursuit of profit; but the VC class seems to desire that anyway, so I don't think we can assume that a free market would consider that a bad outcome.

That VC class appears to really enjoy the frisson of bullshit, elaborate games of guess-what’s-behind-the-curtain, and status posturing. Remove that hedonistic factor and the optimization is likely to be much more effective.

A fair amount of evidence has existed for at least 50 years that a chimpanzee throwing darts at a wall can outperform most active fund managers, yet this has done nothing to reduce their compensation or power.

It has been known since the 1920s that capitalism isn't perfectly efficient. The competition has always been between an imperfect market directed by distributed human compute vs a planner directed by politicians directed by human computed.

It is an argument about signal bandwidth, compression, and noise.