Why do tree-based models still outperform deep learning on tabular data? (2022)

Re:Why do tree-based models still outperform deep learning o... by hackerlight on Tuesday March 05, 2024 @10:31AM (#39604651)

> this would likely require an additional tensor channel, so that each "feature" carries information in a high enough dimensional space

Suppose input data is [batch_size, num_features]. Then you do x.unsqueeze(1) giving you [batch_size, num_features, 1]. Then what?

[ Parent | on HN ]

Re:Why do tree-based models still outperform deep learning o... by hansvm on Tuesday March 05, 2024 @11:34AM (#39605807)

You probably want something equivalent to (however you make it fast in your chosen framework):

einsum('bf,fc->bfc', batched_inputs, channel_embedding)

Then carry that info through the network and project it down at the end. It's roughly equivalent to the token embedding step in an LLM.

[ Parent | on HN ]