Re:Snowflake Computing raises $263M for its data warehouse
by dang on Monday January 29, 2018 @03:31PM
(#16260399)
Could you please don't use uppercase for emphasis? The site guidelines ask you not to: https://news.ycombinator.com/newsguidelines.html.
I'll mark your comment editable for a while if you want to correct this, and can delete this comment if you do.
Re:Snowflake Computing raises $263M for its data warehouse
by georgewfraser on Monday January 29, 2018 @11:46AM
(#16258569)
What makes you think snowflake is more expensive? When I have tested it, it has given higher performance / $. Have you seen different results with your workload?
Re:Snowflake Computing raises $263M for its data warehouse
by sol_remmy on Monday January 29, 2018 @01:31PM
(#16259515)
If I remember correctly, Redshift became more cost-effective over Snowflake the larger your compute got.
Snowflake would just run an EC2 instance in the background for compute, and the very largest EC2 instances were more expensive to run than a Redshift cluster of similar power.
Re:Snowflake Computing raises $263M for its data warehouse
by georgewfraser on Monday January 29, 2018 @01:50PM
(#16259660)
Snowflake is most definitely running multiple instances for anything but the smallest clusters. It's plausible that one warehouse might scale better as you increase the number of nodes but I would really be surprised if there was a big difference---the techniques for query execution are pretty well-known, so Redshift and Snowflake are probably doing the same things. I did a benchmark I while ago [1] and I'm definitely planning on adding multiple scales to evaluate how they compare as you increase data size.
[1] https://blog.fivetran.com/warehouse-benchmark-dce9f4c529c1
Re:Snowflake Computing raises $263M for its data warehouse
by MechanicalTwerk on Monday January 29, 2018 @12:11PM
(#16258822)
I'm going to get downvoted to all hell but come on? All caps? Scales up forever?
Re:Snowflake Computing raises $263M for its data warehouse
by sol_remmy on Monday January 29, 2018 @01:34PM
(#16259530)
I say unlimited because concurrent reads were basically a non-issue for Snowflake. You could have as many users reading from the same table as you wanted.
Meanwhile "Redshift enforces a query concurrency limit of 15 on a cluster." http://blog.blazeclan.com/what-is-amazon-redshift-11-key-poi...
When a Redshift cluster gets too big: 1. Takes forever to take backups, create read-replicas, modify anything 2. Redshift clusters can still only handle X concurrent queries, no matter how big the cluster is 3. Also Redshift is dragging their feet on adding new features i.e. Active Directory support.
Snowflake clusters allow a theoretically unlimited amount of users to query the same tables in a database - so it scales up forever.
And each team can create their own "compute cluster" which allows teams to track and divide costs much more easily then trying to associate this-batch-user on Redshift with this-team/line-of-business.
Snowflake is well-designed for an enterprise environment where line-of-businesses need to share/access each others data but still have separate infrastructure. Redshift does not make it easy go between Redshift clusters.
Also Snowflake as a company is still hungry and is willing to add features if you pay enough
EDIT: However Snowflake is more expensive than Redshift
EDIT: Splitting data between two Redshift clusters was not efficient for us and trying to have the same table in multiple Redshift clusters requires you creating your own tooling and the data might not stay in sync so the only real option was keeping everything in one massive cluster