Re:Ask HN: BigQuery vs. Redshift vs. Athena vs. Snowflake
by vgt on Tuesday March 21, 2017 @09:15PM
(#13927796)
Happy to help!
On partitioning - sounds like a great candidate for sharding. BigQuery standard SQL makes it very easy to run unions on shards (ex. Select * from tables_*)
On pricing - BigQuery is a very different product functionally, and I encourage you to compare it via a poc. That said, definitely reach out to me on price.
These two issues particularly worry me:
- No non-date partitioning, which means you'll often end up (in my particular use case) scanning 10-100x as much data as you actually want to.
- Pricing is based on uncompressed data size, which is at least several times more expensive than the alternative (e.g. Athena).
Any thoughts on these? We'll have several hundreds of TB of data, so cost matters.