This phenomenon has been shown repeatedly in studies of trading behaviour among individual investors (e.g. [1] and [2]), who have been found to collectively destroy wealth for all other investors. Only an absolutely minuscule proportion of traders actually beat the market.
As with any casino, there are winners and losers. But usually the house wins.
[1] https://faculty.haas.berkeley.edu/odean/papers%20current%20v...
[2] https://www.sciencedirect.com/science/article/abs/pii/S13864...
Stop comparing traders and active funds in your study citations. That's not the benchmark. I'm not a fund (which has its own issues as it has to manage 100s of millions which is different than an account like one of ours), and I'm not a day trader/trader. Buying and holding stocks is hardly a zero sum game. You're confused with day trading and options trading which DOES have two sides of a bet. There's something about humanity that some folks here fail to understand when their noses are in the numbers. Are you trying to sell to me the idea that buying AAPL in the early 2000s was a random pick? I'm done here, all that's happening is I lose points as punishment for questioning orthodoxy.
Simple: you got lucky in a bull market.
Let's revisit how you're doing in the next recession or after a couple bad bets.
If you want a deeper answer you'll have to do your own digging, as it's a big topic. But it's worth starting with the efficient markets hypothesis and reading some of the work of Jack Bogle.
There's something strange about the framing of the passive index fund scenario. Over 20-30 years? I'm not interested in beating that benchmark if I'm interested in increasing my net wealth in the next few years. Downvote away, this is my experience and feeling on the topic, not financial advice to others.
Index funds are high risk! What if there's a big dip and you find the perfect house?
Bond rates are low and a bit uncertain at the moment. What's your 'cash replacement'?
Other winners include tech companies and a space company.
Someone explain to me why I'm an imbecile and why I should have been invested in Vanguard index funds, something I did for decades prior to thjs.