News for hackers. Stuff that matters.
Re:Do Machine Learning Models Memorize or Generalize? by ComputerGuru on Thursday August 10, 2023 @05:58PM (#37082303)
> but that's exactly what OP said, no?

Not precisely. We don’t know if verbatim capacity is limited (and it doesn’t seem to be) but the brain operates in a space-efficient manner all the same. So there isn’t necessarily a causative relationship between “memory capacity” and “means of storage”.

> Likewise, if you remembered every single word in a book, you would not understand its meaning

I understand your meaning but I want to clarify for the sake of the discussion that unlike with ML, the human brain can both memorize verbatim and understand the meaning because there is no mechanism for memorizing something but not processing it (i.e. purely storage). The first pass(es) are stripped to their essentials but subsequent passes provide the ability to memorize the same input.

Re:Do Machine Learning Models Memorize or Generalize? by whimsicalism on Thursday August 10, 2023 @06:40PM (#37082642)
> verbatim capacity is limited

I am but a simple physicist and I can already tell you it is.

Re:Do Machine Learning Models Memorize or Generalize? by ComputerGuru on Thursday August 10, 2023 @09:48PM (#37084149)
I mean in terms of our ability to reach those limits, naturally.
Re:Do Machine Learning Models Memorize or Generalize? by SanderNL on Thursday August 10, 2023 @06:32PM (#37082580)
We know for certain it is limited. Do brains not adhere to physics?
Re:Do Machine Learning Models Memorize or Generalize? by astrange on Sunday August 13, 2023 @07:33AM (#37108929)
They only live 100 years and the write rate is only so high.