News for hackers. Stuff that matters.

A comment on Apache iceberg the Hadoop of the modern-data-stack?. Parent comment

Re:Apache iceberg the Hadoop of the modern-data-stack? by bloomingkales on Thursday March 06, 2025 @10:37AM (#43281416)
Something like Redis instead? [sensorid-timerange] = value. Your key is [sensorid-timerange] to get the values for that sensor and that time range.

No more files. You might be able to avoid per usage pricing just by hosting this on a regular vps.

[ Parent | on HN ]
Re:Apache iceberg the Hadoop of the modern-data-stack? by Gasp0de on Thursday March 06, 2025 @10:53AM (#43281614)
We use Redis for buffering for a certain timeperiod, and then we write data for one sensor for that period to S3. However we fill up large Redis clusters pretty fast, so we can only buffer for a shortish period.
[ Parent | on HN ]