Open Source Datasets

Re:Open Source Datasets by nerdponx on Friday June 16, 2017 @10:49AM (#14569483)

OS helps tremendously in reproducibility. Without the underlying data, there is no way to audit an analysis. Moreover, an algorithm is only ever "done" in the same way that software is ever "done". New techniques might come along that could enhance the model, or the business requirements might change that necessitate re-tuning the algorithm.

The benefits of OS data are the same as the benefits of OS software. The distinction between "Free" and "Open" is the same as well.

Edit 1: OS data sets are nothing new. The UCI Machine Learning Repository[1] has been around for years. There is also an entire Open Data Stack Exchange site [2], and an Open Data Subreddit [3].

Edit 2: OS data sets are essential for developing new algorithms because they can be used as benchmarks. Nobody should trust a model that's been developed on a proprietary data set for use on anything other than that one data set.

[1]: https://archive.ics.uci.edu/ml/

[2]: https://opendata.stackexchange.com/

[3]: https://reddit.com/r/opendata/

[ Parent | on HN ]

Re:Open Source Datasets by ptero on Friday June 16, 2017 @11:42AM (#14569929)

Maybe I was not clear -- I am not arguing for proprietary datasets. Validation on publicly available data is a key component in comparisons, assessments, etc.

However, there is a whole bestiary of open source licenses that span the spectrum of "use any way you want" to much more restrictive. But they were mostly thought through for software and data is different; what may prevent proprietary abuse in software may not have any teeth for data.

[ Parent | on HN ]