I can't speak for the other regions but in ap-southeast-2 we had a couple of incidents in 2019 where there was no Spot capacity for ANY instance types between say 64GB and 512GB of RAM.

And so even our instance fleets were failing to provision capacity.

Requiring a minimum of 64GB RAM seems quite unique computationally, so if it's mission critical that you have at least one instance of this type all the time up, then maybe you could reserve that one instance fully while depending on spots for scaling?

this is why ability to fall back to on-demand instances is critical.

When there are no spot instances, it's common that on-demand instances can't be started either.

Unless you paid for a reservation, you're SOL.

Fwiw, Product@SpotInst here. Curious what makes you arrive at this conclusion? At SpotInst we have customers who migrate between Spot <--> RI <--> OD everyday.

that's for you to decide how many percent of reserved instances out of total compute power you need. it's always a trade off. in that case, pct of cost saving is lower but availability is higher.

It seems weird that AWS wouldn't just do this automatically, right?

Yes and no. Because they do not want to disclose how much capacity they have and on demand costs more, customers would be left wondering if AWS unfairly decided to run on demand when spot could have been used instead.

I'm also puzzled why people wouldn't just use spot fleet with on-demand as backup.

because at the time when we did this (2016-2017), kops did not support spot fleet. worst case scenario, you get mix of reserved + ondemand instances.

I don't think I've ever seen spotfleet not satisfy a request with all the same instance type. When the instance gets terminated, it seems to fill again with the same instance type as before (and then get killed again in busy periods).

For the past few weeks I've been having this exact problem with GPU instances, where on-demand requests would kill my spotfleet instances in the middle of a job, I'd spin up again, get killed again a few minutes later.

The only way to fix it without rewriting a bunch of stuff was to blacklist that instance type from my spotfleet requests.

It definitely wasted a bunch of my time though which seems to be increasingly more common with newer AWS APIs.

Fwiw, Product@SpotInst here. Might I suggest that you give us a try. We ward off against this specific scenario by calculating a Spot score based on Instance Type/AZ and deliver flexibility with a choice of different instance types.

because at the time when we did this (2016-2017), kops did not support spot fleet