This is what I find most exciting, ever since I ran across the story of Superforecasters, who supposedly develop techniques to narrow down the probabilities of specific future events within very accurate bounds.

https://en.wikipedia.org/wiki/Superforecasting:_The_Art_and_...

A lot of work has to be done on overcoming cognitive biases and accounting for black swan events in order to attain high level performance.

The point of black swan events is that you can't account for them. Prediction is irrelevant because you cannot compute the probability of an event that has never occurred before. That is why you care about robustness, prediction is inherently fragile.

I have read most of Taleb's books and I'm aware of his objections (and public beef with Tetlock).

Nonetheless, empirically some people are better than others at the task of assigning probabilities to events (as insanely hard as it is!) at a level above random, and it's a learnable skill that gets better with more people.

https://commoncog.com/blog/forecasting-under-uncertainty/

Yep, I worked in finance for years (investing capital) and I have bet on sports profitably for years. My point was actually nothing to do with superforecasting but superforecasting is obvious bullshit. Probabilities aren't computable in any consistent way, and the nature of these events mean that it is often very difficult to work out if someone's track record is actually real (example: the sample size you need for events that occur infrequently is, in practice, much larger than the sample size you need for events that occur more frequently). In the real world, all that matters is information. The idea that there is a group of some super-smart genius forecasters is a rather common mistake of academics...in the real world, the issue is not having information (i.e. the outcome of a vote is not a random event, that is why investment banks/hedge funds pay politicians for information). Btw, I think philosophers in the late 18th century understood this...suggesting that you can sit in your ivory tower and just reason through all these things should seem ludicrous.

Also, you cannot predict black swans. If you think you can, you don't understand what black swan is. The point of a black swan is that it is unpredictable because it has never occurred before.

> The idea that there is a group of some super-smart genius forecasters is a rather common mistake of academics

I'm pointing out is that there do exist some people who are more accurate than the others at assigning probabilities consistently, year after year. This is the real world phenomenon that needs explanation (though see below for caveats).

> Also, you cannot predict black swans

You don't have to exactly predict black swans to roughly estimate probabilities of events. As in, you don't need to enumerate exactly what could happen in the future, you do need to be aware of the possibility but some events are more likely than others.

Now, Taleb would say this is all nonsense because you can't meaningfully deduce a probability of a future event without knowing the denominator, that is look at all future possible worlds leading to outcome A as the numerator and (all future possible worlds leading to outcome B + worlds leading to A) as the denominator.

Which is strictly speaking very true and we are liable to fool themselves. Yet at the same time you can deduce that on aggregate people above the age of 80 have fewer years to live than people under 8... and if you were to predict how long someone would live based on their age, would you just throw your hands up and say it's impossible to know because of black swan events? Or might you look at actuarial tables and try to come up with a probability, especially if there's profit to be had in the form of selling insurance...

Taleb would counter that these predictions are fragile, that one day a black swan will come along and steamroll you when the next cosmic gamma ray event wipes out only the young and not the old, wiping out the past 200 years of gains.

Is that what you're referring to when you say that (super)forecasting doesn't work?

No, you can account for Black Swan events: Talib's whole company is built around the idea that they are more probable than people assume.

The Superforecaster approach absolutely should account for this, and to some degree does. You won't see anyone who has done superforecaster training saying things like "It is 99% likely X will win a two horse race election" before any votes have been cast.

I thought the real take away was that we (humans) have an inherently bad understanding of the true nature of randomness. He suggests that a 'Black Swan' is still a 'Black Swan' (completely new, and totally unpredictable), but there are also classes of events that we 'think' we can ascribe a probability to them, and others where we actually can.

The problem is in conflating the two.

His investment strategy, as I understand it, is to capitalize on that confusion. Which is exceptionally hard work, in spite of a summary of Taleb's idea.

No, read his books. He says repeatedly: you cannot model the probability of these events, his strategy is to increase robustness to Black Swan events.

I'm somewhat of a fan of Taleb but I've never quite figured out how this works out in practice.

1. Taleb says the stock market systematically under-prices risk when in reality probabilities have fat tails, or high kurtosis.

2. Taleb took advantage of this by buying OTM puts which were being undervalued compared to the real risk. He's now "F you" rich, awesome.

(correct me if I'm wrong on these two assumptions)

The next question I have is, now that the cat's out of the bag, does this strategy still work? Is it possible that the market could over-value tail risk, making it a losing proposition to run this strategy (at least on certain stocks).

And if so, how do you know when that deep OTM put option is over or under priced?

I recall Taleb spent a lot of time talking about the problems but was light on actual details of how to make this strategy work out in practice (though it's been a few years since I've read some of his earlier books).

Hopefully you will see this a few days after the fact.

Markets, and people generally, don't work like that. The exact logic varies with strategy but, in this case, people like strategies that involve picking up quarters in front of a steamroller (i.e. selling deep OTM options).

These strategies sell exceptionally well with retail investors. French investment banks, in particular, are known for absolutely loving these strategies. And they blew up hundreds of thousands of Korean investors a couple of years ago with this stuff (and they did this in 2008 too...people really love these strategies and French banks really love blowing up retail investors).

A slightly more trenchant example is tech stocks. These are high momentum stocks, we know that high momentum is extremely cyclical and will eventually blow up (i.e. like option selling) but people do it because they think they will be able to jump out before it goes wrong (spoiler: they won't).

So yes, there is always a price at which something stops working but it is very unlikely we are at that point with this strategy. Implementation of this strategy varies. Empirica (or whatever they are called) apparently do something clever, I know someone doing this who just buys 10-15% OTM puts (and does some kind of simple backtesting/valuation)...but the basic idea is: what is the probability of the price falling this much, what do I gain if this hits, what do I lose if it doesn't. You can get very complex with modelling volatility, if you are trading volatility then that is relevant...for this strategy, you have no real idea when it is going to occur (the PM I know who was buying puts was doing it for three or four years before this happened, they still a lost a bit in the drop but, importantly, they didn't have to sell anything so have had a great year (up 10-15% with very little net exposure to markets).

The training that good judgement project gives you is a beautiful introduction to all sorts of evidence weighing, prediction, black swans, power dynamics of an outcome etc. Should be required for more professions and possibly democracies