Re:Giving meaning to 100B analytics events a day with Kafka,...
by manigandham on Wednesday April 18, 2018 @04:11PM
(#16870717)
Agreed. The pricing is definitely problematic for constant use at smaller companies. We had a single query that read 200GB of data once an hour, and this ends up costing around 700/month. That's for 1 dashboard chart. Add up to 10 charts and we're looking at 7k/month just to run a single company report updated hourly.
BigQuery has high throughput but also very high latency and the shared tenancy means unpredictable query times, with the same SQL taking 10 seconds or 60. They are working on both but it'll be awhile before any changes. DML merge and DDL statements are now in beta though which resolved other big obstacles in automation.
It's a great system but at this point I'd only recommend it for occasional but heavy queries that need to scan massive datasets with complex joins or for some ETL uses. Perhaps if they charged for compressed storage and scans then it would be different but right now Snowflake Data is a much more usable system day to day.
BigQuery has high throughput but also very high latency and the shared tenancy means unpredictable query times, with the same SQL taking 10 seconds or 60. They are working on both but it'll be awhile before any changes. DML merge and DDL statements are now in beta though which resolved other big obstacles in automation.
It's a great system but at this point I'd only recommend it for occasional but heavy queries that need to scan massive datasets with complex joins or for some ETL uses. Perhaps if they charged for compressed storage and scans then it would be different but right now Snowflake Data is a much more usable system day to day.