Lessons Learned from Benchmarking Fast Machine Learning Algorithms

Re:Lessons Learned from Benchmarking Fast Machine Learning A... by alexbeloi on Tuesday July 25, 2017 @07:08PM (#14852618)

In decision trees more than half of the optimization time is spent doing sorting (sorting the set at each node to find the optimal splitting for the key at that node), in neural nets it's almost all matrix multiplies. That's where the speedup difference comes in the CPU v GPU comparison.

[ Parent | on HN ]

Re:Lessons Learned from Benchmarking Fast Machine Learning A... by yorwba on Wednesday July 26, 2017 @12:52AM (#14853723)

There are fast parallel sorting algorithms that should be able to take advantage of GPUs. Maybe they didn't implement them?

[ Parent | on HN ]

Re:Lessons Learned from Benchmarking Fast Machine Learning A... by hoaphumanoid on Wednesday July 26, 2017 @01:31AM (#14853846)

The LightGBM implementation on GPU is based on this paper: https://arxiv.org/abs/1706.08359 they use several smart techniques to make the computation faster. One is how they create histograms of features that are computed in parallel in the GPU

[ Parent | on HN ]

Re:Lessons Learned from Benchmarking Fast Machine Learning A... by folli on Wednesday July 26, 2017 @05:13AM (#14854657)

You need to sort it once for each dimension/predictor, after that it's log(n) to find the optimal split.

[ Parent | on HN ]

Re:Lessons Learned from Benchmarking Fast Machine Learning A... by insulator on Wednesday July 26, 2017 @02:01AM (#14853940)

actually, the histogram-based tree learning algorithm doesn't need the sorting. The main cost of it is the memory access. The GPU version is slow (when #data is small) due to the additional memory copy cost between GPU and CPU. I think this issue can be fixed in the future.

[ Parent | on HN ]