Large-Scale Transactional Data Lake at Uber Using Apache Hudi

Re:Large-Scale Transactional Data Lake at Uber Using Apache ... by cmollis on Wednesday June 10, 2020 @09:00AM (#23476531)

We're about to evaluate it since it's incubating now in AWS' EMR. I think if you have existing spark workloads that require update or delete of existing data stored in parquet, then it might be a good choice since it fills in a major gap there. The other choice is Delta Lake which provides similar capability.

[ Parent | on HN ]

Re:Large-Scale Transactional Data Lake at Uber Using Apache ... by xyzzy_plugh on Wednesday June 10, 2020 @10:25AM (#23477328)

I believe Netflix's Iceberg (now also Apache) aims to solve the same problems.

[ Parent | on HN ]

Re:Large-Scale Transactional Data Lake at Uber Using Apache ... by hashhar on Wednesday June 10, 2020 @10:36AM (#23477436)

Iceberg solves more problems than what Hudi does. The biggest win is computed partitions and not having to define a strict partitioning strategy from the start.

[ Parent | on HN ]

Re:Large-Scale Transactional Data Lake at Uber Using Apache ... by sub3421 on Thursday June 11, 2020 @12:04AM (#23485092)

Unlike hudi and delta lake, iceberg does not support updates/deletes yet.

[ Parent | on HN ]