News for hackers. Stuff that matters.

A comment on Understanding Parquet, Iceberg and Data Lakehouses. Parent comment

Re:Understanding Parquet, Iceberg and Data Lakehouses by snthpy on Saturday December 30, 2023 @12:51AM (#38813071)
Thanks for this. I've been following this space for about a year or two and was wondering why Iceberg was more popular in open source.

Over the past six months I got the impression that Delta is pulling ahead in the race as Iceberg is struggling to provide tools for people not in the JVM ecosystem. Delta is a lot more accessible in that way.

[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by wenc on Saturday December 30, 2023 @12:55AM (#38813090)
DuckDB (lightweight, non-JVM, many language bindings) supports querying from Iceberg now.

https://duckdb.org/docs/extensions/iceberg.html

You still need Spark to generate the Iceberg metadata though.

[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by nerdponx on Saturday December 30, 2023 @03:29AM (#38813679)
Snowflake is rolling out Iceberg support and not Delta support, I think that says a lot.
[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by apwell23 on Saturday December 30, 2023 @07:18AM (#38814701)
Bigquery too.
[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by jamesblonde on Saturday December 30, 2023 @12:54AM (#38813084)
I guess you are referring to delta-rs (for Python in particular). An interesting factoid here is that Databricks started delta-rs, and other companies are now driving it forward - not Databricks. I guess it is not in Databricks interest to push the non JVM ecosystem. PyIceberg is catching up. Write support is almost there - https://github.com/apache/iceberg-python/pull/41
[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by alexott on Saturday December 30, 2023 @01:45AM (#38813270)
As I remember, delta-rs was started by Scribd, not by Databricks: https://youtu.be/2jgfpJD5D6U, https://youtu.be/scYz12UK-OY
[ Parent | on HN ]
Re:Understanding Parquet, Iceberg and Data Lakehouses by jamesblonde on Saturday December 30, 2023 @02:38AM (#38813460)
I stand corrected, then.
[ Parent | on HN ]