News for hackers. Stuff that matters.
Re:Why do tree-based models still outperform deep learning o... by dcl on Tuesday March 05, 2024 @06:00PM (#39610211)
Regression targets where extrapolation may be needed. Decision tree methods cannot extrapolate, the predictions are have to be a mean of a subgroup of the data.

Consider: Predicting how much a customer might pay by end of month, with information we have at the start of the month.

In this example, if a customer had a record $10m of open invoices due by EoM and the largest payment amount received in prior months of $5m, the decision tree cannot possibly predict the payment amount will be ~$10m, even when the best feature indicates the payment will be $10m.

There are some hacks/techniques which can maybe reduce this issue, but they don't always work.

Re:Why do tree-based models still outperform deep learning o... by melondonkey on Wednesday March 06, 2024 @09:54PM (#39624576)
What? Can you explain the mechanism than a NN can “extrapolate” an invoice where a tree model couldn’t? This is all just how the modeler builds the features.

Also all models are a “mean of the subgroup of the data.” The prediction is by definition the conditional mean as a function of the input values.