Parquet: More than just “Turbo CSV”

Re:Parquet: More than just “Turbo CSV” by cm2187 on Tuesday April 04, 2023 @09:28AM (#35439248)

To be honest any file format that would do 1) strong typing/binary data, 2) string deduplication would capture most of the advantages of parquet, except being a column store format which helps in some cases and hurts in others.

[ Parent | on HN ]

Re:Parquet: More than just “Turbo CSV” by julik on Tuesday April 04, 2023 @05:33PM (#35446127)

And even using a columnar format but without all the complexity of Parquet would be comparatively easy (when writing using buffers)

[ Parent | on HN ]

Re:Parquet: More than just “Turbo CSV” by nomadluap on Tuesday April 04, 2023 @07:22PM (#35447183)

At that point just start using an SQLite database as a data format.

[ Parent | on HN ]

Re:Parquet: More than just “Turbo CSV” by jonstewart on Tuesday April 04, 2023 @08:32PM (#35447701)

SQLite is a row-oriented store, not a column-oriented store, ie, there are fundamental tradeoffs between the two.

[ Parent | on HN ]

Re:Parquet: More than just “Turbo CSV” by cm2187 on Tuesday April 04, 2023 @10:21PM (#35448489)

But sqlite has no compression by default.

[ Parent | on HN ]