Re:CPU-based algorithm trains deep neural nets up to 15 time...
by guywhocodes on Thursday April 08, 2021 @08:43PM
(#26745545)
This is the initial SLIDE paper, this is supposed to be an improvement on top of SLIDE. Having looked at and run the original implementation I'm a lot less skeptical than the average post here. It was quite fast already and should be considered a toy implemenation.
[ Parent
| on HN ]
Re:CPU-based algorithm trains deep neural nets up to 15 time...
by foerbert on Thursday April 08, 2021 @10:06PM
(#26745999)
I think the reason you're getting so many questions about the 'toy' statement is that often toy programs are simple in large part because they omit handling complex or difficult cases. I don't think I've ever heard the term applied to a complete problem that is merely implemented naively.
As a result, saying it's a toy implementation is making people think you mean the speedup is a result of simply handling a simple portion of the problem, rather than thinking you mean even a naive implementation is quite fast.
Re:CPU-based algorithm trains deep neural nets up to 15 time...
by gumby on Thursday April 08, 2021 @08:59PM
(#26745668)
Did you mean, "should not be considered a toy" or "this original was a toy/PoC and this new paper is much better"?
Re:CPU-based algorithm trains deep neural nets up to 15 time...
by guywhocodes on Thursday April 08, 2021 @09:02PM
(#26745685)
> this original was a toy/PoC and this new paper is much better
This one hundred percent, the original has very few optimizations over a completely naive implementation. It uses MPI and huge pages, that's essentially it.
Re:CPU-based algorithm trains deep neural nets up to 15 time...
by djrogers on Thursday April 08, 2021 @08:52PM
(#26745614)
What do you mean by ‘toy implementation’?
Re:CPU-based algorithm trains deep neural nets up to 15 time...
by guywhocodes on Thursday April 08, 2021 @08:56PM
(#26745640)
PoC for fun