If someone had a machine that could predict the stock markets with 100% reliability, or even 55% reliability, do you think they would make those predictions known to the public? Or would they start a hedge fund and become one of the wealthiest individuals of all time?
It isn't a whole lot different than an FFT which pickts apart frequency components, HFT algorithms amplify correlation and dampen noise. Then they trade on the correlated signal against the affected securities in an anticipation of the correlated signal's effect.
If correlation was equal to causation they would be 100% effective at making lots of money, but that isn't. So to the extent that the algorithm identifies a correlation that is close enough, it makes money. But it can be crazy like 'sunny days in Paris' is correlated with higher shopper volume in at 'The GAP' and as the sunny days begin to pile up the algorithm buys GAP shares and as the sunny days decline it can sell off GAP shares.
That example we can imagine that people think about buying new clothes when it is sunny but the algorithm doesn't care what the correlation is, just that it has a strong probabilistic link to the changing value of a security. Nothing magical about it.
Nowadays, though, you see this sort of behavior in legal and more respectable ways. However, I think the purported results are highly disingenuous. What do I mean? Let's use a well known investment advisory service known as AAII.
AAII has a model shadow stock portfolio with fairly well known and transparent stock selection criteria. However, they often announce stock picks and stock sells to their followers within a day, supposedly, of their actual transaction. I've been studying the behavior of their stock pick/sells announcements, and on average, stocks are trading +5% to -5% (depending on whether it was a buy or sell) within the day following the announcement. This is HUGE for many reasons:
1) They often have multiple transactions on the portfolio in any given year.
2) The performance on the portfolio is boosted by, on average, 5% for any buy or sell. Compounded, this makes a big statistically significant difference on their purported shadow stock portfolio's CAGR and strategy.
3) They do not disclaim this behavior as a caveat emptor.
4) Though their strategy is transparent, their data feed / source is not. No one will be able to replicate their stock picks/sells on their own with their own live data source.
Quant shops have to continually research to detect new patterns to get ahead of the competition for this reason.
I mean, imagine there is a machine that is capable of predicting the markets with 100% reliability. Then soon enough, the machine will acquire a reputation and everyone will follow its advice. The machine says "the price of X will go up", everybody would buy X, so the price of X would indeed go up, but this price surge would have only been initiated by the machine statement and we'd have a self-realizing prophecy. Other situations would be more complicated I suppose, but in every case the machine would have to take into account its own prediction, which would be some kind of a mise en abyme.
In the end just like antic oracles, the machine would have to make cryptic predictions that would make sense only with hindsight.