The other problem is scale - this is pretty evidently an advertising attempt in order to raise more cash. It's a lot easier to return 20% on, say, $100M AUM than it is on $1B AUM.
To be fair -- these caveats are true for almost every fund you hear about. I do think that's kind of my big issue with the article though, that this fund is just like every other fund; it's intrinsically an actively managed portfolio with the same shortcomings as any other. It's just got a good publicist!
Unless there is a way to break this data down much more finely (by 6-day window, by individual asset) in which case it's possible there's a high p-value but low-power effect. Even so, if there are more than 100 experimental firms, a single one with p-value .01 isn't evidence enough to jumping to big conclusions.
Maybe you'd weight it based on class size? 4 left isn't bad in a class of 100, but it's a bit high in a class of 10.
"Fuzzing" for alpha that way would be really inefficient.
If that's all you're doing, you might as well go all-out on claiming to have secret sauce that produces your alpha. I'm not expecting you'll get any, just that you might as well have a media-friendly way of picking if you don't have an actually useful one.
How do you define an "attempt", and how many "attempts" do you need to beat the market (consistently) by chance?
If you're going to quantify survivorship bias, you can't use entire years as data points, because that doesn't properly represent the amount of activity that occurs. We should reason about each event, because if consistency emerges on an event basis we might not even need more than one year for our sample. The decision-making that is being empirically examined here (i.e. acumen capable of beating the market beyond chance) ostensibly functions on trading events, which means years are not the correct data point to use (and will provide an incorrectly pessimistic sample).
If the data is reported on a yearly basis then it's pretty much the same thing.
I can quibble about the odds of each individual trade resulting in profit or less being binary, but for the sake of argument it'll do. But a 50% chance of beating the market each year isn't supported by anything.
The grouping of data reporting doesn't suggest anything about the underlying data if it doesn't also share the same probability distribution. The trades are the events which determine if a fund will outperform on an annual basis, and we can group those trades by day, week, month, year, etc.
Strategy B: buy two random stocks.
If the expected payoffs were mathematically different then you'd have an arbitrage opportunity. Then apply induction.
Wow, so they are ~27% off the mean!
The part that isn't clear is how this compared to typical performance variation - then we can run our p-tests. But I suspect its pretty good, given the news worthiness of the article.
Also for the actual return:
2013 +22%
2014 -2%
2015 +33%
2016 +22%
2017 +10% (til June)
(1.22 * 0.98 * 1.33 * 1.22 * 1.1) ^ 0.2 = 1.16Or for the 4 whole years:
(1.22 * 0.98 * 1.33 * 1.22) ^ 0.25 = 1.18