The payoff from AGI may be incalculable, but it isn't infinite both in itself and in Altman's ability to enjoy the rewards it promises. Once the value becomes finite, a whole heap of risk-reward logic kicks in that the Wager wants to sweep under the rug.
As a concrete example, following this Altman's wager would result in Altman giving all his wealth to the first beggar on the street who mumbles that he might be able to run an AGI project - the possibility that the beggar can achieve that is, technically speaking, nonzero. Multiply that by infinity and you have a great expected return (infinite, in fact). However, practically speaking, the risk will overwhelm the large-but-finite payoff.
Infinity is bigger than people think :P
Religious beliefs can be weird.
1. The government could change the rules (i.e. redistribute wealth) in response to the new economic reality brought about by AGI
2. The AGI may decide to retain the economic value it captures for itself
I get the feeling that some people (maybe not this commenter) are missing the point I'm making.
Yes, Pascal's wager is flawed - which is why I'm modeling the Altman's wager after it.
I thought I had made the wording and respective payoff statements sufficiently tongue-in-cheek, but I guess this is Poe's law biting back.
(also, as much as I'm also tongue in cheek when talking about AGIs punishing [infinite numbers of simulations of] Sam Altman for backing the wrong AGI project, I'm dead serious about them potentially being able to turn a decent profit selling advanced data processing to Silicon Valley startups. And let's face it, Sam's a much better CEO to be able to help them achieve that than AI Rapture anyway)
Actually makes a bit more sense from the traditional view that AGI projects might not reach their goal but the well run ones are likely to have very commercially valuable byproducts anyway. If we were getting Star Trek economics out of it, who'd be interested in entirely obsolete concepts like "economic value" and "100x returns" anyway?