Re:Google Cloud is 50% cheaper than AWS
by vgt on Sunday November 20, 2016 @04:41PM
(#13001421)
Great comments!
Another way to look at Dataflow pricing as it related to this type of technology. Let's just assume for a second Dataflow and other similar technologies execute your pipeline on your data within the same resource-time amount.
Dataflow lets you set an upper bound on these resources, and lets you auto-scale, with essentially per-minute granularity. Alternatively, you can create a cluster of fixed resource amount. Cost of Dataflow is $0.01 per hour per CPU, in addition to those resources.
Dataflow's model hence either guarantees 100% resource efficiency, or you specifically opt into the upper bound of spend. A deployment model that lets you "spin up a cluster and pay for it" is intrinsically less efficient.
I wrote a blog on this WRT BigQuery, but Dataflow is right there [0].
Re:Google Cloud is 50% cheaper than AWS
by estefan on Monday November 21, 2016 @06:36AM
(#13004612)
Thanks. I know it's tricky due to scaling, etc, but I guess in the same way dataflow works out when data should arrive by (this is a streaming task btw), an estimate of the daily cost could be calculated and would be useful.
Another way to look at Dataflow pricing as it related to this type of technology. Let's just assume for a second Dataflow and other similar technologies execute your pipeline on your data within the same resource-time amount.
Dataflow lets you set an upper bound on these resources, and lets you auto-scale, with essentially per-minute granularity. Alternatively, you can create a cluster of fixed resource amount. Cost of Dataflow is $0.01 per hour per CPU, in addition to those resources.
Dataflow's model hence either guarantees 100% resource efficiency, or you specifically opt into the upper bound of spend. A deployment model that lets you "spin up a cluster and pay for it" is intrinsically less efficient.
I wrote a blog on this WRT BigQuery, but Dataflow is right there [0].
[0] https://cloud.google.com/blog/big-data/2016/02/understanding...
(Work on Google Cloud)