Re:Snowflake’s response to Databricks’ TPC-DS post
by sagarm on Wednesday November 17, 2021 @01:41PM
(#29257064)
The _data_ should be replicated, but the compute infrastructure doesn't need to be. Many companies I suspect would be fine having to restart pipelines on driver failure (increasing tail latency, basically) if it yields a substantial cost reduction.
If I'm reading what Databricks published correctly, it seems that they've only used 1 driver node for this benchmark, in other words it's a dev setup. If they want to compare apples-to-apples then they should configure, and price, a multi-AZ HA set-up.
I'm not sure if this is still applicable to Photon, however - can anyone confirm?