News for hackers. Stuff that matters.
Re:When your data doesn’t fit in memory: the basic techniques by kyllo on Tuesday November 12, 2019 @11:41AM (#21515477)
This is time partitioning, not indexing, but it's commonly done in data lakes for raw telemetry/log data.

    mydata/
        2019/
            11/
               12/
                   2019111200.csv
                   2019111201.csv

Where each one of those CSV files could be 10-100 GB in size.

Usually you want to process it into a columnar format like Parquet from there, though

There are no comments on this comment yet.