Interesting experiment and a good start at evaluating the concept, in particular, the fact that they compared to other methods makes it way stronger than similar concepts that forget to even compare to index. However, even though they clearly tried to avoid obvious pitfalls of backtesting, they also left lots of potential sources for bias.

Their method of collecting news is based on what popped up in their post-facto web search, it also relied on them classifying them for relevance. This can generate bias compared to, for instance, feeding all articles tweeted from a specific news source irregardless of "relevance". I also wonder what assumptions they've made about timing, slight errors in timing estimated could have huge impact on results.

The fact that the shorting seemed to make for the most part of the gains (although the long strategy was not terrible at 50% gain vs index going down) might also be something somewhat biased towards the fact that this was a downturn year and it might not fare as well in a more regular year.