That was a very elegant way to make it click for me, thank you.
How is that the theoretical ideal when a few good stocks far outperform a maximally diversified portfolio?
The longer explanation is that you can break down the price effect of various real-world events into two axes. There are things that transfer market value between companies, and things that change the value of the market as a whole. For example, people buying more Macs instead of PCs will transfer market value from Microsoft (among others) to Apple. On the other hand, a novel form of efficient energy production will do more to increase overall economic activity.
The maximally market-cap weighted portfolio has diversified away the first axis of risk. Suppose company A, worth $500M, loses $50M of market cap to company B, worth $100M. If you own $500 of company A and $100 of company B, the $50 loss from owning company A stock is completely offset by owning $10 of company B stock. You're purely exposed to changes in the value of the economy as a whole, and have eliminated diversifiable risk from your portfolio.
If you can't benefit from taking on diversifiable risk, you're better off getting the same return at a lower level of risk by simply removing it.
Market cap lost to non-public companies will disappear from your portfolio.
The problem isn't that the non-random strategy did poorly, it's that it has too much exposure to needless sources of uncertainty. The inverse strategy doesn't get rid of uncertainty, it just flips the sign of their exposure to that uncertainty.