News for hackers. Stuff that matters.
Re:CPU-based algorithm trains deep neural nets up to 15 time... by guywhocodes on Thursday April 08, 2021 @08:43PM (#26745545)
This is the initial SLIDE paper, this is supposed to be an improvement on top of SLIDE. Having looked at and run the original implementation I'm a lot less skeptical than the average post here. It was quite fast already and should be considered a toy implemenation.
Re:CPU-based algorithm trains deep neural nets up to 15 time... by foerbert on Thursday April 08, 2021 @10:06PM (#26745999)
I think the reason you're getting so many questions about the 'toy' statement is that often toy programs are simple in large part because they omit handling complex or difficult cases. I don't think I've ever heard the term applied to a complete problem that is merely implemented naively.

As a result, saying it's a toy implementation is making people think you mean the speedup is a result of simply handling a simple portion of the problem, rather than thinking you mean even a naive implementation is quite fast.

Re:CPU-based algorithm trains deep neural nets up to 15 time... by gumby on Thursday April 08, 2021 @08:59PM (#26745668)
Did you mean, "should not be considered a toy" or "this original was a toy/PoC and this new paper is much better"?
Re:CPU-based algorithm trains deep neural nets up to 15 time... by guywhocodes on Thursday April 08, 2021 @09:02PM (#26745685)
> this original was a toy/PoC and this new paper is much better

This one hundred percent, the original has very few optimizations over a completely naive implementation. It uses MPI and huge pages, that's essentially it.

Re:CPU-based algorithm trains deep neural nets up to 15 time... by djrogers on Thursday April 08, 2021 @08:52PM (#26745614)
What do you mean by ‘toy implementation’?
Re:CPU-based algorithm trains deep neural nets up to 15 time... by guywhocodes on Thursday April 08, 2021 @08:56PM (#26745640)
PoC for fun