(https://www.newyorker.com/tech/annals-of-technology/chatgpt-...)
Lossy compression = Intelligence
That's where the Hutter Prize falls down, it's based on lossless compression, which is nothing like how the brain works.
Not precisely. We don’t know if verbatim capacity is limited (and it doesn’t seem to be) but the brain operates in a space-efficient manner all the same. So there isn’t necessarily a causative relationship between “memory capacity” and “means of storage”.
> Likewise, if you remembered every single word in a book, you would not understand its meaning
I understand your meaning but I want to clarify for the sake of the discussion that unlike with ML, the human brain can both memorize verbatim and understand the meaning because there is no mechanism for memorizing something but not processing it (i.e. purely storage). The first pass(es) are stripped to their essentials but subsequent passes provide the ability to memorize the same input.
I am but a simple physicist and I can already tell you it is.
On the other extreme, are packers. They have optimized for packing facts in bulk, with little regard for how they fit together. If you give this type of person a set of instructions that require a wider knowledge of how things fit, they will get lost, frustrated, and/or need support. If you anticipate this, and spend a bit extra time to show how to handle all of the possible contingencies, (and give them a document of this) they're good, and will be quite happy with your support.
I think that mappers take more time figuring out the model, compressing the facts to save space, and increase applicability in general.
...but that's exactly what OP said, no?
I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget"
If we memorized everything perfectly, we would not learn anything: instead of remembering the concept of a "chair", you would remember thousands of separate instances of things you've seen that have a certain combination of colors and shapes etc
It's the fact that we forget certain details (small differences between all these chairs) that makes us learn what a "chair" is.
Likewise, if you remembered every single word in a book, you would not understand its meaning; understanding its meaning = being able to "summarize" (compress) this long list of words into something more essential: storyline, characters, feelings, etc.