Re:BTables: A fast, compact format for Machine Learning
by glogla on Friday October 02, 2015 @06:41PM
(#10321760)
I wonder if it weren't more practical to just use sqlite file for data like that. I mean, it's not plaintext, but sqlite is available pretty much anywhere and provides convenient interface for data.
[ Parent
| on HN ]
Re:BTables: A fast, compact format for Machine Learning
by felixr on Friday October 02, 2015 @06:55PM
(#10321818)
HDF5 is much better suited for a lot of scientific data sets. How would you store multidimensional data in sqlite? Not everything is a table or matrix.
HDF5 also allows you to pick compression filters that are especially suited for the data you have.
If you are looking to replace a CSV file, then sqlite is obviously a pragmatic solution.