The true distribution of the market beliefs in future stock prices can be understood by empirically studying the volatility smile [0]. That is, because investors know Black-Scholes is not a perfect mathematical model of real world stock behavior, every strike price has a different implied volatility. By looking at these different IVs you can get a sense of what the market believes are the true probabilities of "long tail" events.
In theory, the opportunities you have to make money should be cases where you believe the market has mispriced risk. In my amateur experience, I have found that virtually every time you think the market has mispriced some extreme event, when you look at the volatility smile, you realize you are mistaken.
1) where there are pretty complete markets for implied volatility, looking at the past matters less to little, because there is a market for the "future volatility" you can hedge and interact with
2) when there isn't a good volatility market and hedging future volatility exposure is difficult, looking towards the past for some guidance increases in importance
Both things can get complicated at times and in both cases it isn't strictly speaking the stddev you care about, but the quadratic variation (which can be the same under some assumptions).
You take a view on volatility by buying or selling an option, if you are right then you will make money proportional to the options gamma (i.e. the convexity of the option is where the money comes from)
Actual volatility (not implied!) is much easier to predict than price.
It’s also much more difficult to trade than price changes. So your intuition about this is correct though.
It is not super difficult to predict tomorrows volatility sign (up/down compared to today) with +60% success. Even textbook GARCH models do well here.
If you could do that with the price, you’d quickly become filthy rich.
there is possibility you can take trading position with expectation of vol reverting to the mean, and vol will keep increasing (what happened to tesla and gamestop short sellers)
and vice versa
When trading the VIX, you are trading the implied volatility not the actual (realized) volatility.
VIX represents the implied vol of options on S&P500 expiring 30 days into the future.
Trading the realized volatility is not easy :)
Lots of options trading involves taking a position on whether you think that implicit estimate is too high or too low. Generally, a long options position encodes belief that volatility is cheap and visa versa. Options are also a very specific kind of instrument and can be used to craft very specific bets on volatility. For instance, you might feel that the options at a $200 strike are pricing too high of an implied volatility compared to those at the $195 and $205 strikes.
Traders build an intuition around the model instead of treating it as in and of itself predictive. They instead try to price or take bets on certain derived quantities from it (the "greeks").
The saying goes that implied volatility is "the wrong quantity put into the wrong model in order to make the right decision".
Black-Scholes/Merton makes a lot more sense once you work it all out yourself in code.
I'd actually suggest doing this through modeling the underlying geometric Brownian motion and ensuring that your simulated results match up to the analytic formula.
Option Traders can quote each other in IV without disclosing their asset pricing models and assumptions (trade secret tech).