News for hackers. Stuff that matters.
Re:Fault Tolerance Benchmark: Clockwork TorchPass, TorchFT a... by essekar on Wednesday March 11, 2026 @01:43PM (#47338730)
One of the feedbacks we got while testing was - we might even reduce the duration of checkpointing. Which was a huge insight.

We are testing out & would love more collaboration from the community - if your team is running training jobs, hit us up.

Re:Fault Tolerance Benchmark: Clockwork TorchPass, TorchFT a... by gdcohen on Wednesday March 11, 2026 @03:44PM (#47340287)
I agree. If you can reliably recover from failures by migrating instead of restoring from checkpoints, then the need to checkpoint frequently becomes far less important and reduces the overhead of taking checkpoints. Similarly, a lot of users are experimenting with asynchronous checkpoints to reduce the blocking penalty, but there are big tradeoffs there with DRAM usage. So being able to take checkpoints far less frequently gives you more flexibility on the type of checkpointing you use as well.