Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Spot Instances

Re:Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Sp... by ignoramous on Sunday October 03, 2021 @01:40PM (#28738947)

Neat. Congratulations on the launch, Vishnu!

Apart from the fact that it could deploy to both GCP and AWS, what does it do differently than AWS Batch [0]?

When we had a similar problem, we ran jobs on spots with AWS Batch and it worked nicely enough.

Some suggestions (for a later date):

1a. Add built-in support for Ray [1] (you'd essentially be then competing with Anyscale, which is a VC funded startup, just to contrast it with another comment on this thread) and dbt [2].

1b. Or: Support deploying coin miners (might help widen the product's reach; and stand it up against the likes of consensys).

3. Get in front of the very many cost optimisation consultants out there, like the Duckbill Group.

If I may, where are you building this product from? And how many are on the team? Thanks.

[0] https://aws.amazon.com/batch/use-cases/

[1] https://ray.io/

[2] https://getdbt.com/

[ Parent | on HN ]

Re:Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Sp... by vishnukool on Sunday October 03, 2021 @02:21PM (#28739286)

Thank you! Interesting, we actually tried aws batch ourselves. 1) How were you able to handle spot interruptions and resuming from the latest checkpoint ? 2) Not to mention fallback to OnDemand on spot interruptions 3) then switching back to spot from onDemand would also need additional process to be setup.

Also i'm not sure how straightforward it is to detach/attach persistent volumes to retain data across different spot interruptions ? The latter can be done but it's just the same rote each time you wanna train something new.

Also thanks for the suggestions ! We're a team of 2 right now, I used to be in the bay area but in Mexico temporarily.

[ Parent | on HN ]

Re:Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Sp... by ignoramous on Sunday October 03, 2021 @03:47PM (#28739953)

1. Spot interruptions didn't matter much as AWS Batch looks for spots with low interruption probability. Auto retries kicked-in whenever those did get interrupted.

2. Checkpointing was a pain (we relied mostly on AWS Batch's JobState and S3, not ideal), but the current capability to mount EFS (Elastic Filesystem) looks like it would solve this?

3. No hot swapping on-demand with spot and vice versa. Interestingly, ALB (Application Load Balancer) supports such mixed EC2 configurations (AWS Batch doesn't).

[ Parent | on HN ]