Neural Networks and the Chomsky Hierarchy [DeepMind, 2022]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by guerrilla on Monday January 23, 2023 @04:57AM (#34487098)

> Also the human brain has finite memory.

How can we know that? If memory in some way uses energy levels for encoding then we know it must be finite because of quantization but do we know that?

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by bruce343434 on Monday January 23, 2023 @05:33AM (#34487293)

Why do you think it could be infinite? What physical arguments can you make?

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by guerrilla on Monday January 23, 2023 @05:43AM (#34487351)

I don't think there's any reason to assume that it's finite or infinite. There could be physical properties in real life that can vary continuously. Energy isn't one, but there could be others.

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by BlueTemplar on Monday January 23, 2023 @07:13AM (#34487923)

But none that we know about ?

Taking the SI base units, which can be used to derive all the others that we know about :

Time, Energy ~ Mass and Length are quantified by Heisenberg's Uncertainty Principle.

Temperature doesn't really exist, it's kind of a proxy for information through energo-informatics, quantified, thanks again to the HUP : http://www.av8n.com/physics/thermo/entropy-more.html#sec-pha...

Electric Current by the charges being discrete.

Mole by definition.

Candela is not a physical unit (but one based on human biology).

----

Of course, that's a different question of whether their amounts in our universe can be (countably) infinite : so far as we know, time is (and maybe space, which should make other space-extensive quantities like mass-energy and charges also (countably) infinite ?)

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by guerrilla on Monday January 23, 2023 @08:29AM (#34488484)

This isn't an argument that should convince anyone unless we both have a theory of everything and you want to then enumerate every possible variable involved. Any ignorance would leave a gap. The Bekenstein bound mentioned in the other comment though demonstrates that the approach isn't even necessary to come to the same conclusion though (as far as I can tell anyway.)

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by BlueTemplar on Monday January 23, 2023 @12:30PM (#34491992)

That doesn't seem reasonable to me : we do not need to preface every statement with "with our current understanding of physics", this goes without saying.

We might never get a "theory of everything", and we shouldn't assume the existence of new physical properties without any other reason to do so, this goes against the principle of parsimony ! (Especially continuous ones, none of which seem to exist for now !)

And if our understanding of physics does change, it could have other dramatic consequences - that Bekenstein bound might not be valid any more for instance...

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by guerrilla on Monday January 23, 2023 @12:45PM (#34492233)

We're not assuming the existence of such physical properties. We're also not assuming the inexistence of such physical properties. That's the point. You're making a leap based on no evidence. I'm making no such commitment and your arguments shouldn't convince anyone to make that leap with you. Worse, you want to use this unfounded leap to claim we know something else. It's terrible epistemology. It turns out we do know that other fact, but certainly not by following you down your unsound road.

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by T-A on Monday January 23, 2023 @05:23AM (#34487217)

https://en.wikipedia.org/wiki/Bekenstein_bound

(yes, that's a ridiculously high bar, but still finite)

[ Parent | on HN ]

Re:Neural Networks and the Chomsky Hierarchy [DeepMind, 2022] by guerrilla on Monday January 23, 2023 @05:47AM (#34487369)

I see, that's seems to be a pretty conclusive argument.

[ Parent | on HN ]