Re:Understanding Parquet, Iceberg and Data Lakehouses
by ceencee on Sunday December 31, 2023 @12:12PM
(#38825555)
I feel like this is frankly uninformed. Many iceberg shops seem to rely heavily on Spark as a primary engine. And databricks has a history of being a hostile oss force with the culture of the spark project being toxic from the start and delta’s questionable commitment to being a community project.
The choice between the two resembles the choice between Parquet and ORC circa 2016. Two formats of broadly the same power, initially biased by a particular query engine, eventually at feature parity and universally supported.
We have got a decade of experience with OSS from Databricks so doubting their "open ecosystem" status seems a little theoretical.