Re:Ask HN: How to store billions of rows of time-series data?
by Kkoala on Monday April 06, 2020 @04:45PM
(#22797668)
Rows will be at least (id, other_id, price_1, price_2), prices should be 32 bit integers, id should be 64 bit or string integers and other_id can be 32 bit int. id and other_id should reference other things. Or if normalization is not possible then it would add up to bunch of names and other metadata, 6 strings more at minimum, but more like 10 strings or so.
The data will be a stream basically.
Granularity is quite important, but I think older data could be made less granular and turned into approximates.
The data will be a stream basically.
Granularity is quite important, but I think older data could be made less granular and turned into approximates.