News for hackers. Stuff that matters.
Re:Regularization is all you need: simple neural nets can ex... by jackylupino on Wednesday June 23, 2021 @06:29AM (#27602866)
Batch norm has an advantage for iterative methods on mini-batches, while XGB uses the full training set. Using batch norm on the full training set is equivalent to Z-normalizing the features, which has no effect at all for XGB as the scale of features plays no role at the split decisions of the tree nodes. Apart few non-parametric data augmentation methods (notice adversarial augmentation is also nn specific), I do not think any other regularization used in that paper can be directly/intuitively applied to XGB.

There are no comments on this comment yet.