I believe this is exactly what slot reservations in BigQuery achieve. Instead of paying on-demand pricing that is determined by data read, you purchase a fixed number of “slots” that are shared by queries running within that particular project.
Ah OK, after reading their docs I see they've changed what "slots" used to mean in Dremel (internal version of BQ). It used to be that slots _guaranteed_ capacity, but did not limit it. Meaning that you could rely on having a certain number of workers in the cluster when you issue a query, but if Dremel had more it'd give you all it's got. Obviously this is not viable when people have to pay per terabyte read, because a ton can be read.
What they have now strikes me as an even better solution to the problem of bankrupting someone with a query IMO. Not sure how pricing compares to redshift et al, but pricing is the easiest thing for Google to change.
Re:Why is Snowflake so Valuable?
by manigandham on Thursday October 01, 2020 @04:18AM
(#24648658)
Slots don't control how much data you consume, your query does.
If you need to read a terabyte of data to answer your query then more slots only gets it done faster.
Re:Why is Snowflake so Valuable?
by statusgraph on Wednesday September 30, 2020 @07:53PM
(#24645569)
BQ Slots lets you do essentially that (pre-commit to a particular cluster size)
Also, BQ does explain the query plan to some extent: https://cloud.google.com/bigquery/query-plan-explanation. Not quite at the level of a "regular" SQL DB, but it does give you some info to work with when optimizing queries. If you haven't used it in a while I'd give it another try.