News for hackers. Stuff that matters.
Re:Parquet: More than just “Turbo CSV” by ted_dunning on Tuesday April 04, 2023 @01:11AM (#35435358)
Yes. Ish.

Obviously text-oriented grep doesn't work. But table oriented duckdb can work very well indeed (and it basically combines a lot of awk and grep together)

  $ duckdb <<EOQ
  select count(\*) from 'archive.parquet' where sensor = 'wifi'
  EOQ
  ┌──────────────┐
  │ count_star() │
  │    int64     │
  ├──────────────┤
  │      4719996 │
  └──────────────┘
  $
You can change a lot of aspects around such as the output format, but the point remains that you can do grepy things with Parquet files quite easily.

There are no comments on this comment yet.