insulator

here is a benchmark for catboost:

https://github.com/szilard/GBM-perf/issues/4

The parameter of LightGBM is a little bit hard to tune. And when #data is small, it is easier over-fitting. But it will be powerful if you tuned it well (Here a guide for the parameter tuning: https://github.com/Microsoft/LightGBM/blob/master/docs/Param...).

Also, there are many kaggle winning solutions using lightgbm recently. e.g. The 2nd on "Quora Question Pairs" used the ensemble of 6 lightGBM and 1 NN (https://www.kaggle.com/c/quora-question-pairs/discussion/343...). And almost top-10 in this competition used LightGBM as sub-models for the ensemble.

actually, the histogram-based tree learning algorithm doesn't need the sorting. The main cost of it is the memory access. The GPU version is slow (when #data is small) due to the additional memory copy cost between GPU and CPU. I think this issue can be fixed in the future.

CatBoost is benefited from categorical feature transform. LightGBM is also working for the better categorical feature support (https://github.com/Microsoft/LightGBM/issues/699). I think the accuracy of LightGBM will be comparable with CatBoost when it is finished.