News for hackers. Stuff that matters.
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by anonymousDan on Monday June 13, 2022 @11:51AM (#31727236)
How does the concept of a table here differ from that of a standard relational table (if at all)? Is it that the table is a logical abstraction over a distributed set of files?
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by chrisjc on Monday June 13, 2022 @01:11PM (#31728432)
These are effectively external tables. They likely exist on blob storage and not in any particular DB's storage system. As a result, you have a myriad of ways to interact with that data. Moreover, compute and storage resources are now decoupled.
Re:Comparison of Data Lake Table Formats (Iceberg, Hudi and ... by SpicyLemonZest on Monday June 13, 2022 @01:10PM (#31728401)
The underlying storage is less structured, which means that some common database optimizations (in particular row-level indexes) are unavailable in these kind of systems, but it's conceptually the same as a standard relational table.