News for hackers. Stuff that matters.
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by henrydark on Monday June 13, 2022 @12:10PM (#31727475)
A major problem with these table formats that will surface soon enough is that they use serial numerical ordering for versions.

It's like inventing SVN for data. Soon enough git will have to be invented as well.

Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by mrry on Monday June 13, 2022 @12:20PM (#31727638)
"Project Nessie: Transactional Catalog for Data Lakes with Git-like Semantics"

https://projectnessie.org/

This supports lightweight branches, and transactional commits and merges. I haven't used it—and it seems cool—but it also seems a little heavyweight to get cross-table transactions on top of these table formats (which would be my primary use case).

Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by alexmerced on Monday June 13, 2022 @02:17PM (#31729265)
Project Nessie also powers Dremio's Arctic service, so you can get all the branching and benefits of Nessie with an intuitive UI to browse branches, create branches and merge branches. Also, it is a cloud managed service with a free tier.
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by kristjansson on Monday June 13, 2022 @08:30PM (#31733420)
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by ozkatz on Thursday June 23, 2022 @01:37PM (#31852535)
Check out lakeFS (https://github.com/treeverse/lakeFS). It doesn't rely on the object store for ordering and is highly influenced by Git itself (but designed to work on object stores at very large scales)