News for hackers. Stuff that matters.
Re:Deep learning for twelve hour precipitation forecasts by candiodari on Sunday February 12, 2023 @07:14AM (#34762267)
> Show that your approach works on real-world data, that you didn't have access to when training your model. Show that your models can predict the future, not the past. Otherwise, your experiments don't mean, what you think they mean.

Great, but ... that requires you have collection infrastructure and almost by definition means you can't make it work on a dataset. The way ML research works is like the Kaggle competitions.

You get a dataset. This can be anything, as long as it can be expressed as a tensor (which are vectors, but can have more dimensions). Then split into seen and unseen data. You split into 3 portions/partitions. One for training. One for testing, and one for blind validation (80% training and 10% each for test and validation sets).

In order to simulate what you're suggesting researchers can, and often do, take the most recent datapoints as validation and testing sets.

But extending the data set is an expensive process that generally can't be done by the machine learning researchers themselves. So it is not done.

There are no comments on this comment yet.