News for hackers. Stuff that matters.
Re:Facebook Trains ImageNet in 1 Hour by randyrand on Tuesday June 13, 2017 @02:41AM (#14543224)
this seems like the trivial, most obvious way to parallelize training across GPUs. Not, imo, clever.

The important bit here is that they've shown that large mini batch sizes still can maintain accuracy if you slow the learning rate.

Re:Facebook Trains ImageNet in 1 Hour by opportune on Tuesday June 13, 2017 @10:27AM (#14544914)
Just because it's simple to explain at a high level doesn't make it trivial.

Plenty of theoretically trivial solutions to problems are absolute pains to implement. I mean there are entire companies that at their core solve relatively "trivial" problems but employ huge numbers of engineers. Just because the core concept is simple to explain doesn't mean it's easy.

Re:Facebook Trains ImageNet in 1 Hour by randyrand on Tuesday June 13, 2017 @02:44PM (#14547426)
Someone asked the other day how to parallelize gpu learning. I had never thought about the problem before, but still came up with and gave this as the most obvious way.

https://news.ycombinator.com/item?id=14510146

Re:Facebook Trains ImageNet in 1 Hour by willvarfar on Tuesday June 13, 2017 @09:01AM (#14544393)
Lots of clever things are simply self-evident in hindsight.
Re:Facebook Trains ImageNet in 1 Hour by randyrand on Tuesday June 13, 2017 @02:39PM (#14547384)
Someone asked 5 days ago on HN how to parallelize gpu learning. I had never thought about the problem before, but still came up with and gave this as the most obvious way. Took me maybe 20 seconds to think of.

https://news.ycombinator.com/item?id=14510146

So the fact that someone with little experience can come up with this 'clever' technique, means either I'm really clever or it's not that clever. I'll go with the latter.

Re:Facebook Trains ImageNet in 1 Hour by Seanny123 on Tuesday June 13, 2017 @04:40AM (#14543569)
very personally, clever means "well, that would have taken me a long time to figure out and code properly"