News for hackers. Stuff that matters.

A comment on Ask HN: How do I improve our data infrastructure?. Parent comment

Re:Ask HN: How do I improve our data infrastructure? by riku_iki on Saturday April 20, 2019 @10:58AM (#19706844)
you can run sql DB over compressed filesystem, and some DBs allow you to compress tables too

> like being able to deserialize a single column without having to read in entire rows.

and it reads filesystem's whole page anyway

[ Parent | on HN ]
Re:Ask HN: How do I improve our data infrastructure? by aeroevan on Monday April 22, 2019 @12:31PM (#19720269)
Sorry for the late reply, but parquet is a columnar format so if it's big enough data, you should have multiple pages/blocks of data in a single column for a specific row group, and then be able to seek to the next row group and sequentially read the next set of blocks.
[ Parent | on HN ]