Re:Apache Hudi vs. Delta Lake vs. Apache Iceberg Lakehouse F...
by joeharris76 on Thursday January 12, 2023 @01:16PM
(#34358010)
1) create a table (and then query it) - seems very common
2) merge more into it (and then query it) - also seems very common
3) delete (a tiny bit) from it (and then query it) - also seems very common
4) delete (a little more) from it (and then query it) - also seems very common
5) delete (a decent chunk) from it (and then query it) - also seems very common
6) delete (all data for 1 key) from it (aka "GDPR request") - new problem, but common
I sense a pattern here. Not sure what makes these tests "absurd" since everyone does them.
First, you should probably add disclaimer that you work for Databricks.
Second, they really did not.
They came up with a test case of 1) create a table 2) merge into it 3) delete from it 4) delete from it 5) delete from it 6) delete from it (but call it "GDPR request" in chart).
This is a completely absurd workload that should give anyone a pause. In real world, Lakehouse tables are either appended to, merged into with periodic compaction, or dropped and recreated (using something like dbt). Running a delete four times is just bizarre, and making a general statement out of it is just suspiciously biased.
Of course it is very interesting coincidence that this independent company came up with non-realistic benchmark showing how better Delta is compared to major competitor, that everyone from Databricks is sharing everything.