How is the cap price not on-demand cost? Why would you not just swap to on-demand at the turning point?
This means "bidding" more won't protect your capacity, AWS will just take it if they need it.
Prices are now relatively flat, and changing slightly over time by a fraction of a cent per day with a cap at the OnDemand price, while previously there used to be lots of from 0.1x to 10x the on demand price and with swings even multiple times within a day.
It is because many systems are only built to support spot. When running a workload, they might not want to give up availability so they just pay the higher price. To be fair, this is a game that AWS engineers. They essentially want to kick people off the infrastructure to free it up for other uses. So this is a way to signal how badly you don't want to be kicked off. A lot of times after a price goes above on-demand pricing, it will dip down below fairly quickly as workloads all re-adjust. Some companies are willing to play that gamble, that a short period above on-demand is still cheaper in the long run when you average out the invoice at the end of the month.
Most hours, you’d pay something close to $0.30/hr. Some hours, you’d pay over a $1.00/he, but you’d save money overall against on-demand.
(This ignores Reserved Instances and Savings Plans.)
If you set up an EKS/K8s cluster and install Karpenter. You can configure it very easily to use spot instances anytime prices are available for less than on-demand and to use on-demand when spot instances are unavailable or too expensive.
You end up never thinking about it but having full availability.
In practice it means the ceiling on your bill is the on-demand price, but you usually average out much lower.
I suspect as more and more people switch over to this model then the use of spot instances will stay closer to full saturation, with those discounts becoming negligible.
They have a whole section in their docs about this: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-int...
It's all about how your spot instances can be terminated. Perhaps this could interfere with a workload that might take a long while to finish.
On paper your app should be resilient to these things but in practice it's essentially an unforgiving chaos monkey that will delete servers from your cluster.
If you can handle that, you can definitely save a lot though. Especially web servers where it's expected you'll be responding in a few seconds while maybe your background workers are churning through longer lived tasks. The problem there is you need to already be big enough to have different nodes handling web vs worker traffic. A lot of decently sized apps can operate in a world where maybe you have 2-5 nodes on your cluster and your web and worker pods land on any of them based on Kubernetes' default scheduler.
This does require architecting your services to be shut down tolerant, but that's par for the course. If you're using Kubernetes you've probably already settled on the "[herd, not] cattle, not pets" idea.
If you have heavy duty work being done, it might get kill -9'd after the grace period is surpassed. If you weren't using spot instances you have full control over the grace period.
You can of course handle this too, by breaking up your work in ways where it can be interrupted without losing progress but depending on what you're doing this could be very complicated. Basically the takeaway is you can't blindly start using spot instances, even if your app has been running in Kubernetes successfully for a while.
I have often seen the following:
1. The demand for spot instances increases for a period of time.
2. Supply is inelastic so prices quickly rise to the upper bids.
3. Most users are unwilling to go hours without any spot instances running, so they use very high bids.
You either have to tolerate hours of "down time" every few months, or pay exorbitant prices during once in a while.
On multiple occasions, I've seen the spot instances priced significantly higher than on-demand.