News for hackers. Stuff that matters.
Re:Databricks open-sources Delta Lake to make data lakes mor... by cavisne on Wednesday April 24, 2019 @11:44PM (#19744786)
The number of definitions here suggest its not well defined, one more :)

A data lake is when you store your data in object storage (S3, GCS etc) as opposed to a filesystem (HDFS) or some indexed datastore (Redshift etc).

This potentially saves a lot of money because you can scale compute separately from storage, and object storage can be extremely cost effective compared to running a distributed filesystem.

Where the two overlap is when you store something like parquet in object storage, the file format is somewhat indexed already so you spend a bit more money preprocessing it but save a lot of money querying it.

I think whether its "raw" json or log files or preprocessed parquet doesnt really differentiate whether its a data lake or not

Re:Databricks open-sources Delta Lake to make data lakes mor... by kakoni on Thursday April 25, 2019 @04:55AM (#19746239)
But interestingly delta.io only supports HDFS currently.