News for hackers. Stuff that matters.
Re:220,000 cores and counting: largest ever public cloud job by throwaway2016a on Thursday April 20, 2017 @02:49PM (#14159623)
Can someone explain Google Preamble Instances vs Amazon Spot Instances?

The author says Sutherland was dissuaded by having to specify a spot price upfront on AWS but I don't see how that is any different than what Google is doing.

Re:220,000 cores and counting: largest ever public cloud job by user5994461 on Thursday April 20, 2017 @03:02PM (#14159752)
AWS Spot Instances are under bid. The highest bidder takes the instances, the price changes all the time.

Google Spot Instances (preemptibles) are 80% off and that's it. It's simple.

Re:220,000 cores and counting: largest ever public cloud job by takeda on Thursday April 20, 2017 @06:44PM (#14161564)
Can't you just bid for 20% of the original price and get the same behavior?

In AWS you simply tell how much you're willing to pay to keep the instance uninterrupted. If there's someone who is willing to pay more, they will get the instance from you and yours will be shut down.

Re:220,000 cores and counting: largest ever public cloud job by boulos on Friday April 21, 2017 @12:25AM (#14163088)
With Spot, if the market price goes above .2x retail, and your job runs long enough then you never succeed with Spot.

By comparison, if you are able to create a preemptible VM you are guaranteed to pay .2x retail and then likely (but not guaranteed) to have 24h to do your work.

tl;dr: A maximum bid of .2x retail isn't equivalent to preemptible. There's almost certainly a bid for a given instance for a given runtime that would result in the same price, but it varies over time, space, runtime and instances shape.

Disclosure: I work on Google Cloud (and launched preemptible VMs)

Re:220,000 cores and counting: largest ever public cloud job by williamstein on Tuesday April 25, 2017 @05:33PM (#14198275)
> likely (but not guaranteed)

In my several years experience using GCE it is indeed extremely likely. Two years ago it wasn't very likely, but it became very likely about a year ago. Incidentally, I'm the guy that convinced Drew Sutherland of this article to use GCE in the first place :-).

Re:220,000 cores and counting: largest ever public cloud job by jjn2009 on Thursday April 20, 2017 @08:02PM (#14161986)
The main factor is the market for this unused CPU time, a bid of 20% could be vastly different on each service. The GCE pricing model in a way has a sort of step function applied to the pricing, whereas AWS you can bid however you like, how this effects consumer usage I would guess is higher cost on AWS due to perceived competition. This is all assuming the CPU time on machines is constant.
Re:220,000 cores and counting: largest ever public cloud job by Eridrus on Thursday April 20, 2017 @09:09PM (#14162296)
AWS also tends to have different prices in different regions and for different sized machines, so there's more work involved in making your 20% work.
Re:220,000 cores and counting: largest ever public cloud job by throwaway2016a on Thursday April 20, 2017 @03:19PM (#14159918)
That does sound pretty nice.
Re:220,000 cores and counting: largest ever public cloud job by jfroma on Thursday April 20, 2017 @03:39PM (#14160070)
We run specific components of our application in spot instances across different regions and from what I have seen in the last 3 months is that the price is generally much less than 80%. I'd say is less than 20% most of the time, but some times there is a peak and it goes beyond 150% for 1 to 3 hours, the frequency of these events may vary but I'd say once a week.

I think AWS might be cheaper in this regard but less predictable, so it is a tradeoff.

Edit: I haven't looked yet into "Spot Fleets".

Re:220,000 cores and counting: largest ever public cloud job by joshuamorton on Thursday April 20, 2017 @03:43PM (#14160132)
Not 80%, 80% off.
Re:220,000 cores and counting: largest ever public cloud job by jfroma on Thursday April 20, 2017 @04:21PM (#14160469)
Wops, sorry for my confusion!

Then it is definitely interesting :)

Re:220,000 cores and counting: largest ever public cloud job by vacri on Thursday April 20, 2017 @09:27PM (#14162403)
Looking at the spot prices on our AWS account (us-east-1), most of the time you're looking at around 25% full price. Some instance types seem to go lower than this (r4 is about 12%) but it looks like the 'resting point' is around 25% for the ones I skimmed.
Re:220,000 cores and counting: largest ever public cloud job by jacques_chester on Thursday April 20, 2017 @07:14PM (#14161737)
Preemptibles are at a fixed discount to the normal rate. Spot instances fluctuate according to demand.

That said, there is still a "market" here: your work vs Google's work. It just shows up as a private floating probability instead of a public floating price.

Most of the time you get the full 24 maximum window.

edit: wording

Re:220,000 cores and counting: largest ever public cloud job by boulos on Thursday April 20, 2017 @08:11PM (#14162029)
Just a small correction: it's not about "your work vs Google's", it's about regular VMs versus preemptible. There's no special code to give Googlers (or Alphabet Characters) special treatment with regards to preemption. The chrome clusterfuzz folks are standing side by side with you.

Disclosure: I work on Google Cloud (and launched Preemptible VMs).

Re:220,000 cores and counting: largest ever public cloud job by richardw on Thursday April 20, 2017 @09:17PM (#14162338)
Great answers, thank you.

Economically, because the price is fixed and everyone has the same status, wouldn't that have the effect of oversubscription if that price is always below amazon spot price? Are you eg legally prevented from using a spot-like bid? I'd imagine spot would balance supply and demand better.

Re:220,000 cores and counting: largest ever public cloud job by boulos on Thursday April 20, 2017 @10:55PM (#14162773)
No, we could have done an auction-based pricing model (Google's ads business is clearly a big fan). But I pushed for flat rate and won ;). However, just a note that you don't get oversubscription but rather a "first come first served" result which has its own downsides.
Re:220,000 cores and counting: largest ever public cloud job by asher_ on Friday April 21, 2017 @04:28AM (#14163901)
Excellent! One of GCPs many advantages (outside of it's technical ones) is it's simpler pricing model.
Re:220,000 cores and counting: largest ever public cloud job by jacques_chester on Thursday April 20, 2017 @09:58PM (#14162551)
My suspicion is that Google's unit economics for compute and network are better than Amazon's, because they've never historically had the ability to push costs onto a customer. It all fell to their own bottom line.

If they can sustain a lower price, picking a fixed discount gives them a marketing edge, even if AWS would come out lower overall. Most humans will pay a premium for certainty -- the certainty effect or "taxicab effect".

Re:220,000 cores and counting: largest ever public cloud job by boulos on Thursday April 20, 2017 @10:47PM (#14162744)
There are certainly particular instances in particular regions or zones that are cheaper than the preemptible price we've set. If you look at larger instances, we often are drastically cheaper. Add in the per-minute versus per-hour pricing, and some customers have seen massive wins (I recall one at nearly 10x less).

The distinction, as you've surmised, is that predictability is awfully useful. This 220k run (and Drew's quick 400k run last Sunday) had a predictable price. I doubt that such a run on Spot would have left the market price untouched ;).

That said, we absolutely have excess capacity that maybe having an auction clearing mechanism would fill. But I don't think it's worth the customer pain. Moreover, even Drew has come to us from Spot, suggesting that simplicity at a usually-fair/good price can take market share away (and reminder this is a fast growing market!).

Re:220,000 cores and counting: largest ever public cloud job by jacques_chester on Friday April 21, 2017 @07:00AM (#14164461)
Right.

The basis of budgeting and capacity planning is that everybody who participates in a free market secretly wishes they didn't have to.

Re:220,000 cores and counting: largest ever public cloud job by ohstopitu on Friday April 21, 2017 @11:55AM (#14166664)
quick question about preemption instances:

if I were to launch an instance of Kubernetes (via container engine) with such VMs, would it have an issue? (as in, if I had more load, more preemptible vms would be provisioned automatically based on a previously set max amount and if a vm was removed, another would be assigned based on my minimum amount)

am I wrong in thinking that this is a much better option than running dedicated vms?

Re:220,000 cores and counting: largest ever public cloud job by jacques_chester on Thursday April 20, 2017 @09:53PM (#14162525)
I did not realise that. On reflection it makes sense.
Re:220,000 cores and counting: largest ever public cloud job by kyrra on Thursday April 20, 2017 @03:00PM (#14159731)
Looks like there are a number of differences:

https://www.quora.com/What-are-the-key-differences-between-A...