News for hackers. Stuff that matters.
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @04:09PM (#27596534)
If you want to use NNs on tabular data look up the work that's being done on point clouds. They both share the same major symmetry over permutations.
Re:Regularization is all you need: simple neural nets can ex... by lurkmurk on Tuesday June 22, 2021 @07:09PM (#27598497)
They don't? One is a row in table (R^n) (order matters) and the other is a set of points (R^nxd).

If you consider the whole table, any dataSET is like point clouds.

Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @07:16PM (#27598572)
The table is the point cloud the row is the point. The symmetry is permutations of rows or permutations of points.
Re:Regularization is all you need: simple neural nets can ex... by lurkmurk on Tuesday June 22, 2021 @07:27PM (#27598693)
In which case would you use the whole table?
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @08:13PM (#27599101)
Well yeah, there's no permutation symmetry between columns. (Unless there is...)
Re:Regularization is all you need: simple neural nets can ex... by elcomet on Wednesday June 23, 2021 @02:25AM (#27601402)
Then as parent said, almost all datasets are like point cloud, except time-series datasets.
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Wednesday June 23, 2021 @10:13AM (#27604778)
Yeah, they are.
Re:Regularization is all you need: simple neural nets can ex... by eachro on Tuesday June 22, 2021 @05:24PM (#27597418)
Oh that's interesting. Can you say more? Haven't seen much relating the two topics before.
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @05:50PM (#27597666)
A row in a table of data is an point in R^n. I'm not sure how much there is to write about it other than to say, that's a point cloud.
Re:Regularization is all you need: simple neural nets can ex... by eachro on Tuesday June 22, 2021 @09:21PM (#27599600)
what's the intuition for why that might be desirable? I can sort of see that you might care to consider the relation between a given row and other rows (not disimilar to something like kernel methods) and then you can use something like Deep Sets[1] to featurize the data?

[1] https://arxiv.org/abs/1703.06114

Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Wednesday June 23, 2021 @01:24AM (#27601056)
I think the way it works is, you have one network that produces global permutation-invariant (maintained so by training loss) metrics and another that recognizes based on those metrics. The big prior you're putting in is that the order of the points doesn't matter. Relationships between points do matter but only in a permutation-invariant way. I would recommend reading the literature because of course, it's not my idea. :)
Re:Regularization is all you need: simple neural nets can ex... by asdf3331 on Tuesday June 22, 2021 @07:01PM (#27598428)
No, that's a point.
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @07:05PM (#27598461)
The table's the point cloud. I guess if you want to be really pedantic about grammar then I should point out that a set of one item is still a set. :-)
Re:Regularization is all you need: simple neural nets can ex... by asdf3331 on Tuesday June 22, 2021 @07:33PM (#27598753)
True - I misread your post. Your first post was intriguing but the second was dismissive. Surely there's more to be said than data sets are point clouds? Images are points in R^N too, right?
Re:Regularization is all you need: simple neural nets can ex... by whatshisface on Tuesday June 22, 2021 @08:12PM (#27599092)
If your unit of recognition was a set of images, and not an image alone, then you would have permutation symmetry and want to use point cloud techniques to design your first layer. So yes images are points in R^N.
Re:Regularization is all you need: simple neural nets can ex... by elcomet on Wednesday June 23, 2021 @02:26AM (#27601416)
Oh thanks, this was not clear from your other posts: the whole table is used as a data point, not the line. Much clearer now, why you would compare this to point clouds
Re:Regularization is all you need: simple neural nets can ex... by karmickoala on Saturday June 26, 2021 @05:58PM (#27646313)
Which work do you suggest?