Apart from the fact that it could deploy to both GCP and AWS, what does it do differently than AWS Batch [0]?
When we had a similar problem, we ran jobs on spots with AWS Batch and it worked nicely enough.
Some suggestions (for a later date):
1a. Add built-in support for Ray [1] (you'd essentially be then competing with Anyscale, which is a VC funded startup, just to contrast it with another comment on this thread) and dbt [2].
1b. Or: Support deploying coin miners (might help widen the product's reach; and stand it up against the likes of consensys).
3. Get in front of the very many cost optimisation consultants out there, like the Duckbill Group.
If I may, where are you building this product from? And how many are on the team? Thanks.
[0] https://aws.amazon.com/batch/use-cases/
[1] https://ray.io/
Also i'm not sure how straightforward it is to detach/attach persistent volumes to retain data across different spot interruptions ? The latter can be done but it's just the same rote each time you wanna train something new.
Also thanks for the suggestions ! We're a team of 2 right now, I used to be in the bay area but in Mexico temporarily.
2. Checkpointing was a pain (we relied mostly on AWS Batch's JobState and S3, not ideal), but the current capability to mount EFS (Elastic Filesystem) looks like it would solve this?
3. No hot swapping on-demand with spot and vice versa. Interestingly, ALB (Application Load Balancer) supports such mixed EC2 configurations (AWS Batch doesn't).
No one asked, but it's HN so I'll say it anyway. I think it's a questionable "vc business" but a great business for 1-2 people. The road from this to an enterprise sales motion, or even a 10K/year contract is hard for me to imagine. At some point, it becomes cost effective for my org to build this functionality in house.
However, as a hobbyist /single dev / small team, $120/Year is a no brainer after the first 2-3K I spend on GPU by mistake. As you know, setting up spots when I just want to get shit done is a pain and I'll gladly pay you (a little) to make that go away.
Still no one asked but... One thing that plays to your advantage is that the current price point is something anyone in your user group can buy on their own, and there are a lot of us / enough to make a nice business out of.
Good luck!
So useful (and potentially lucrative) that AWS/GCP would likely take over this functionality eventually -- either by building on top of spot instances like you have, or underneath regular instances or re-use that capacity for some other managed service (thereby reducing the price differential). How do you plan to protect yourselves against that?
Congrats on the idea and godspeed. You’ll probably have a lot of interest if you execute well.
Spot Instances are 70% cheaper than On-Demand instances but are prone to interruptions. We mitigate the downside of these interruptions through the use of persistence features, including optional fallback to On-Demand instances. So you can optimize workflows according to your budget and time constraints.
History: We were working on a neural rendering startup that needed a lot of GAN training which was getting very expensive. We were blowing roughly $1000, to train a single category class. Training on Spot instances was cheaper, but still a mess. It needed lot of hand holding/devops stuff to make it usable. So we built SpotML to automate a lot of things.
Posting it here to see if the community finds this helpful, so that we can open it up to the larger community.