News for hackers. Stuff that matters.
Re:MIT computer scientists can predict the price of Bitcoin by leeber on Friday October 24, 2014 @04:17PM (#8505728)
Agreed. According to the paper: They trained the data, once, with 3-4 months of data from Feb to May. Then they tested the data, once, with ~6 weeks of data from May 6 - June 24.

There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive.

No interesting conclusions can be taken away from this paper due to flawed methodology and failure to take into account real world variables like bid-ask spread, commissions, how quickly trades can be executed, whether orders will even be filled, etc.

Re:MIT computer scientists can predict the price of Bitcoin by beejiu on Saturday October 25, 2014 @12:06PM (#8508548)
Cross validation too uses training sets and test sets. This sort of time-ordered data will not be independent, so the prequential approach seems to be a more suitable approach to measuring forecast accuracy. (I haven't read the paper.)
Re:MIT computer scientists can predict the price of Bitcoin by leeber on Saturday October 25, 2014 @02:00PM (#8508858)
Cross validation for time series, goes something like this:

http://robjhyndman.com/hyndsight/crossvalidation/

(near bottom of the article)

Anyways, as somebody who has spent GIANT amounts of time experimenting with machine learning using market data (mainly stocks), I can give you TONS algorithms I've created that would show similar, even much higher, gains when you only test on 6 weeks of data. Been there done that. For example 6 weeks you make a 100% return, then in the next 3 weeks you suffer a 50% loss. Reality sets in...

Re:MIT computer scientists can predict the price of Bitcoin by vasilipupkin on Friday October 24, 2014 @06:13PM (#8506273)
Well, they did trade it out of sample, so that probably means the model wasn't overfit. But, I agree, they probably used unrealistic execution assumptions