News for hackers. Stuff that matters.

A comment on Databricks is an RDBMS. Parent comment

Re:Databricks is an RDBMS by anonymousDan on Monday February 01, 2021 @08:06PM (#25996333)
Wait what? I thought spark is designed to work on top of a shared nothing cluster? Have you any resources describing the architecture of databricks specifically?
[ Parent | on HN ]
Re:Databricks is an RDBMS by snidane on Tuesday February 02, 2021 @02:37AM (#25998259)
In Databricks you only get cloud storage (s3) as a persistent storage for data. So the data has to move from s3 to compute nodes.

Self hosted HDFS will work more like shared nothing.

[ Parent | on HN ]
Re:Databricks is an RDBMS by anonymousDan on Tuesday February 02, 2021 @03:19AM (#25998466)
Ah,thanks, makes sense.
[ Parent | on HN ]