News for hackers. Stuff that matters.
Re:BTables: A fast, compact format for Machine Learning by glogla on Friday October 02, 2015 @06:41PM (#10321760)
I wonder if it weren't more practical to just use sqlite file for data like that. I mean, it's not plaintext, but sqlite is available pretty much anywhere and provides convenient interface for data.
Re:BTables: A fast, compact format for Machine Learning by felixr on Friday October 02, 2015 @06:55PM (#10321818)
HDF5 is much better suited for a lot of scientific data sets. How would you store multidimensional data in sqlite? Not everything is a table or matrix. HDF5 also allows you to pick compression filters that are especially suited for the data you have. If you are looking to replace a CSV file, then sqlite is obviously a pragmatic solution.