The timeframe was only 1 year, therefore this is a rather inconclusive test. It warrants more investigation, but if a hedge fund understands that feeding good news or bad news to an AI makes the stock market move, eventually they will control the news factory to control the AI.

It is a neat trick though. I can't read that fast and I can't make 500%.

the stock market for short-term traders is a zero-sum game.

if there is a chatgpt trick that produced alpha, it would be adopted by every hedge-fund reducing its effectiveness (and alpha generation).

so, these sorts of tricks depend on suckering retail traders who as a group are the losers. but the stimmies have run out, and so will these tricks.

Interesting experiment and a good start at evaluating the concept, in particular, the fact that they compared to other methods makes it way stronger than similar concepts that forget to even compare to index. However, even though they clearly tried to avoid obvious pitfalls of backtesting, they also left lots of potential sources for bias.

Their method of collecting news is based on what popped up in their post-facto web search, it also relied on them classifying them for relevance. This can generate bias compared to, for instance, feeding all articles tweeted from a specific news source irregardless of "relevance". I also wonder what assumptions they've made about timing, slight errors in timing estimated could have huge impact on results.

The fact that the shorting seemed to make for the most part of the gains (although the long strategy was not terrible at 50% gain vs index going down) might also be something somewhat biased towards the fact that this was a downturn year and it might not fare as well in a more regular year.

ChatGPT is hot news. Professor wants in on the action. He and his grad-student come up with a thesis. Then they cherry-pick evidence supporting the thesis and concoct a catchy headline that makes it to hacker news.

They tested six different strategies and only two-three of the strategies returned profit on this year. Even the ones that turned out profitable were only profitable within the first half of the year and provided no return during the second half of the year.

Seems similar to p-hacking.

>The researchers collated 67,586 headlines pertaining to 4,138 unique companies during this time, using web scraping techniques. These headlines were subsequently filtered for relevance, with the team narrowing their focus to full articles and press releases, excluding any stock-gain or stock-loss headlines. They also removed duplicate news to ensure only fresh information was used.

67,586 headlines over more than a one-year period is under 200 headlines per day. One human analyst could trivially perform sentiment analysis on that many headlines before lunch. Practically perfect sentiment analysis on this scale is thousands of years old.

There is no "special sauce" from ChatGPT.

I have had even better returns with many sided dice. I buy if the number is more than half the maximum and sell otherwise. I back tested over the last year with various simulated die sizes and it turns out a 49652 sided dice returns over 900%!