News for hackers. Stuff that matters.
Re:A promenade of PyTorch by singhrac on Tuesday February 20, 2018 @11:20PM (#16426817)
I wholeheartedly agree about the TF API becoming very cluttered. The entire way contrib is handled seems confusing.

While I think TF probably has a better distributed story, it's still not great (several months of experience getting this to work), and very very few people can justify distributing computation across 10K CPUs/GPUs - in my experience, it turns out most async SGD algorithms don't really work very well.

One place I see this actually working is using SVI/EM and storing local parameters across the network - something maybe 1 GPU can't handle for extremely large models.

There are no comments on this comment yet.