News for hackers. Stuff that matters.
Re:Twitter migrates data to Google Cloud by sandGorgon on Wednesday January 23, 2019 @03:52AM (#18976734)
we are struggling to maintain price parity on Google Cloud versus both Azure and AWS.

AWS now has the new arm machines in production which are cheaper. AWS also allows yearly prepaid managed databases - while Google still charges per second billing (with some monthly discount). My AWS TCO comes out to be much, much lower (almost 30%) than GCP if i take into account committed use/prepayment discounts.

Re:Twitter migrates data to Google Cloud by dijit on Wednesday January 23, 2019 @04:37AM (#18976933)
Interesting; maybe it's not solving the same use-cases for you then :)

For us: on AWS/Azure we had to benchmark each instance when it came up and if it performed poorly it had to be reaped and redeployed constantly. The increased overhead in dealing with things "the AWS way" was enough by itself to offset the pricing difference with the steep AWS discount we were getting.

Understanding administrative overhead is complicated but I'm sure you are taking into account the cost of humans. (or, you've already absorbed the cost of automating it?) :)

As with all things; use what works for you. I'm just very happy with GCP over the others coming from bare metal.

Re:Twitter migrates data to Google Cloud by sandGorgon on Wednesday January 23, 2019 @08:23AM (#18977860)
it depends on usecases - i run a data heavy usecase. We deploy using docker swarm, so our redeployment is fairly well taken care of.

in terms of benchmarking - this is not something we care about (since we dont have so much CPU utilization), so probably its easier for me.

For my usecase, AWS has better administrative tools than google. I use Route 53, SES, S3 - in all those three, no other products come close (transfer acceleration...im looking at you).

The one place that I agree with you is Dataproc. That's far superior to EMR in terms of spin-up/down.

Re:Twitter migrates data to Google Cloud by Ironlink on Wednesday January 23, 2019 @04:40AM (#18976942)
I dislike the concept of committing to use. It creates a disincentive to optimize jobs, because "we've already paid for three years of this".
Re:Twitter migrates data to Google Cloud by pantulis on Wednesday January 23, 2019 @04:44AM (#18976955)
It's up to you to see it the other way around: if you optimize jobs you can fit future workloads in the infrastructure you've already paid for.

I prefer to think of reserved instances as a purely financial trick.

Re:Twitter migrates data to Google Cloud by room271 on Wednesday January 23, 2019 @07:35AM (#18977651)
The pay-as-you-go reductions give you more flexibility about when you make optimisations to your code though and help you handle uncertainty around this.

For example, if I have a new workload that won't fit on existing hardware with AWS, I can either: a) reserve the new instances, or b) not reserve on the assumption I will be able to optimise in the 'near' future.

In practice this is hard to know but AWS forces you to come to a correct decision up front, whereas Google Cloud allows you to not have to answer this up front without being punished later for making the 'wrong' choice.

Re:Twitter migrates data to Google Cloud by Ironlink on Wednesday January 23, 2019 @07:20AM (#18977598)
Right, but that requires your new workloads to fit well with the resources you have committed to. If you've committed to many small machines, they may not be well suited to an especially heavy workload. If you've skewed your commitment towards a large amount of memory per compute (or vice versa), you can find yourself with lots of RAM (or compute) and no natural use-case.

At the bottom line, commitment impedes change.

Re:Twitter migrates data to Google Cloud by pantulis on Thursday January 24, 2019 @04:27AM (#18987185)
Your point is valid, thanks for the answer.
Re:Twitter migrates data to Google Cloud by sandGorgon on Wednesday January 23, 2019 @08:27AM (#18977890)
From a startup POV, I'll give you the other perspective.

I now have a significantly reduced opex that saves me a lot of money. If I grow very fast (and outgrow it)... I really dont care, because I will most likely have a financing event.

If I dont grow fast enough to justify the spend, I have bigger problems.

In addition, I have to commend Google and Azure here - the way they do committed use is very flexible. They price it on number of units (cores, RAM, whatever) - so if you outgrow it, you still get the discount + full cost of additional units. On AWS, if you outgrow, you have to frikking sell off your machines on their auctions and buy new ones.

Only problem is that Google doesnt do this for databases (which are very expensive)

Re:Twitter migrates data to Google Cloud by matwood on Wednesday January 23, 2019 @12:34PM (#18980384)
> On AWS, if you outgrow, you have to frikking sell off your machines on their auctions and buy new ones.

I don't believe this is entirely true. The AWS reserved credits are good within the machine family. So a t2 credit is good for all t2 instance sizes. Not as flexible as CPU/RAM credits, but more so than it was in the past.

Re:Twitter migrates data to Google Cloud by sandGorgon on Wednesday January 23, 2019 @01:07PM (#18980761)
true - but in multiples of instances. so 2xt2.xlarge = 1 t2.2xlarge. But if you have 3xt2.xlarge, then the swap cannot be partially for 2xt2.2xlarge . So you have to sell one of the t2.xlarge.

its not very nice. that's why they have a reserved instance marketplace - https://aws.amazon.com/ec2/purchasing-options/reserved-insta...

what is new is the convertible reserved instances. I havent used it - and you may be right there. But i still maintain the GCP/Azure way is light years better

Re:Twitter migrates data to Google Cloud by matwood on Thursday January 24, 2019 @07:34AM (#18988071)
Which is why amazon’s own tool suggests buying smalls or nanos.

I do think gcp and azure do it better though.

Re:Twitter migrates data to Google Cloud by briffle on Wednesday January 23, 2019 @09:54AM (#18978583)
I like google's implementation of it, since you commit to use a certain amount of CPU, and RAM, and not instance types. So you can commit to a 4cpu/15GB ram system, and a year later, double the size of the vm, and you still have the cheaper price on the first 4CPU/15GB of ram. And its per project. (I would love it if I could purchase per organization)
Re:Twitter migrates data to Google Cloud by manigandham on Wednesday January 23, 2019 @10:40AM (#18979087)
Yes, GCP has the best billing model. You purchase CPU/RAM capacity and it works the same whether on-demand, sustained use discount, or commitments.

Separating capacity from instance type makes it much more natural and easy to use for your actual requirements.