Re:Build your own “data lake” for reporting purposes
by gizmodo59 on Tuesday March 16, 2021 @09:06AM
(#26476183)
There are several advantages compared to CSV, JSON or other non-columnar formats as far as analytics is concerned which are typically not transactional.
And with projects like Apache Iceberg you can also have ACID transactions which will make it easy to update or delete just using SQL. This really opens up the separation of compute and data and you can use any engine you want (spark, drill, hive, impala, athena, dremio, redshift spectrum etc) on top of your files.
https://blog.openbridge.com/how-to-be-a-hero-with-powerful-p... https://stackoverflow.com/a/36831549
And with projects like Apache Iceberg you can also have ACID transactions which will make it easy to update or delete just using SQL. This really opens up the separation of compute and data and you can use any engine you want (spark, drill, hive, impala, athena, dremio, redshift spectrum etc) on top of your files.