News for hackers. Stuff that matters.
Re:Microsoft Zero and DeepSpeed: Memory Efficient Large Neur... by davidwhite on Monday February 10, 2020 @11:21PM (#22296036)
Looks super cool. Does it remove the need for manual gradient checkpointing?

Also curious if there are expected to be memory / speed improvements if you're using it on a single GPU or if most gains come from improved parallelism across devices.

There are no comments on this comment yet.