News for hackers. Stuff that matters.
Re:"time to GPT-2", down to 2.91 hours by hhh on Wednesday February 04, 2026 @05:43AM (#46884160)
it depends, our workloads can finish up in under two minutes and shut down without much effort, so we haven’t really noticed it outside of one time when we had no spot capacity.
Re:"time to GPT-2", down to 2.91 hours by janalsncm on Wednesday February 04, 2026 @05:58AM (#46884287)
I guess if checkpointing is set up correctly and your runtime is saved to a docker image it’s feasible. Probably not going to get a 3 hour continuous chunk of time I would assume.
Re:"time to GPT-2", down to 2.91 hours by direwolf20 on Wednesday February 04, 2026 @06:58AM (#46884801)
When I once used Spot it wasn't that bad. You're likely to have an instance for 3 hours.
Re:"time to GPT-2", down to 2.91 hours by hhh on Thursday February 05, 2026 @08:21AM (#46899353)
I use spot for my gmod server. Contention is higher for GPU, but I only saw disruptions maybe once every 2 months for super common instance types. We scale up and down more frequently than we get disrupted.
Re:"time to GPT-2", down to 2.91 hours by joeig on Wednesday February 04, 2026 @05:59AM (#46884294)
In addition to the two-minute interruption notice, rebalance recommendations[0] allow you to handle interruptions even more gracefully.

[0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/rebalanc...