News for hackers. Stuff that matters.

A comment on Regularization is all you need: simple neural nets can excel on tabular data. Parent comment

Re:Regularization is all you need: simple neural nets can ex... by eachro on Tuesday June 22, 2021 @09:21PM (#27599600)
what's the intuition for why that might be desirable? I can sort of see that you might care to consider the relation between a given row and other rows (not disimilar to something like kernel methods) and then you can use something like Deep Sets[1] to featurize the data?

[1] https://arxiv.org/abs/1703.06114

[ Parent | on HN ]
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Wednesday June 23, 2021 @01:24AM (#27601056)
I think the way it works is, you have one network that produces global permutation-invariant (maintained so by training loss) metrics and another that recognizes based on those metrics. The big prior you're putting in is that the order of the points doesn't matter. Relationships between points do matter but only in a permutation-invariant way. I would recommend reading the literature because of course, it's not my idea. :)
[ Parent | on HN ]