Re:Busting myths about BigQuery
by nothrabannosir on Saturday December 23, 2017 @02:28PM
(#15995785)
In redshift, I control the size of nodes. That means I can brute force my way through an expensive operation by increasing the memory capacity of the nodes. In BigQuery, you play by the rules or you’re out. It forces you to think in map reduce, because there really is nothing else under the hood.
Also, you get immediate parallelisation across O(n) nodes. Again; not in redshift.
These are crucially different to regular DBs. They’re both semantically sql, nobody denies that, but they describe different underlying models.
E.g.: in BigQuery, you can’t sort your entire column, even if you do other stuff afterwards. That makes sense in a map reduce system, but not in a “normal” DB.
For example, if you're writing straight-up SQL, the fewer columns you project out before you start sorting the better off you are, it's better to join stuff after you've done your sort than the reverse because it means physically less data shuffling.