News for hackers. Stuff that matters.
Re:Show HN: Hacker News archive (47M+ items, 11.6GB) as Parq... by vovavili on Wednesday March 18, 2026 @04:04PM (#47430738)
This makes more sense. I still wonder if the author isn't just effectively recreating Apache Iceberg manually here.
Re:Show HN: Hacker News archive (47M+ items, 11.6GB) as Parq... by tomrod on Wednesday March 18, 2026 @04:05PM (#47430751)
Are they paying for the repo space, I wonder?
Re:Show HN: Hacker News archive (47M+ items, 11.6GB) as Parq... by cyanydeez on Wednesday March 18, 2026 @04:25PM (#47430971)
someones paying to keep name dropping Iceberg(tm)
Re:Show HN: Hacker News archive (47M+ items, 11.6GB) as Parq... by mulmen on Wednesday March 18, 2026 @06:40PM (#47432304)
Weird accusation. Iceberg is an Apache project. I don’t think anyone gets paid when you use it so not sure what the benefit of shilling would be. It is just a table format that’s well suited for this purpose. I would expect any professional to make a similar recommendation.
Re:Show HN: Hacker News archive (47M+ items, 11.6GB) as Parq... by tamnd on Thursday March 19, 2026 @06:47AM (#47437294)
I intentionally kept it lightweight. Just Parquet files + simple partitioning + commits on Hugging Face. That already covers most of what I need, without introducing a heavier stack or extra dependencies.

Also, I wanted something that is easy to consume anywhere. With this setup, you can point DuckDB or Polars directly at the data and start querying, no catalog or special tooling required.