Regularization is all you need: simple neural nets can excel on tabular data

Re:Regularization is all you need: simple neural nets can ex... by xiaodai on Tuesday June 22, 2021 @08:26PM (#27599203)

if you spend the same amount of time in xgboost and lightgbm looking for hyperparameters and build trees with a small enough eta. will you achieve the same performance? that's not answered in the paper, and if I am a reviewer I would ask them to come compare time spent.

[ Parent | on HN ]

Re:Regularization is all you need: simple neural nets can ex... by jackylupino on Wednesday June 23, 2021 @06:13AM (#27602779)

After reading the paper I believe they do both aspects that you mention: i) they give XGboost and all the baselines up to 4 CPU days of hyperparameter optimization time on 20 CPU core servers per dataset, same compute as the author's proposed method. ii) the search space for the XGBoost hyperparameters includes low eta (starting from 0.001) and large num_round ranges (up to 1000 trees) in Table 5 at the appendix.

[ Parent | on HN ]