Are deep neural networks dramatically overfitted? (2019)

Re:Are deep neural networks dramatically overfitted? (2019) by mjburgess on Monday April 05, 2021 @05:54AM (#26697085)

One is a subset of the other. Compression being the larger category.

A mean is a compression of a dataset and useful information.

My issue in my comment is that 'compression' corresponds to a massive class of technquies and there isnt a lot of content in the observation that useful information is compressive.

However there is some hypey people out there who think this observation has legs. Precisely the people who think intelligence is a mathematical problem, and not an engineering one -- which is my view.

ie., that a body isnt incidental to intelligence, but the heart of it.

Or: devices matter.

[ Parent | on HN ]

Re:Are deep neural networks dramatically overfitted? (2019) by typon on Monday April 05, 2021 @09:24AM (#26698584)

I tend to think about it the same way as you. Having an algorithm for multiplying two numbers is qualitatively different than having a lossy compression of a huge dataset of multiplication tables. The latter is what GPT3 has and it just doesn't scale.

[ Parent | on HN ]