Why does deep and cheap learning work so well?

Re:Why does deep and cheap learning work so well? by tedsanders on Friday May 11, 2018 @05:07PM (#17050814)

Here's a related 2016 talk by Max Tegmark (second author) on connections between deep learning and physics:

https://www.youtube.com/watch?v=5MdSE-N0bxs

The gist of it is that physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits. Random data does not have this property, and cannot be learned easily. Super fascinating.

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by yodon on Friday May 11, 2018 @05:11PM (#17050842)

Thank you!!! as a Physics PhD that was one of the first videos I found on deep learning, and having no idea what a big deal his insights were I promptly forgot who the speaker was (remembering only it was a name I knew intimately from my time in Physics). Have frequently gone back to try to find this unsuccessfully.

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by 3pt14159 on Friday May 11, 2018 @10:13PM (#17052125)

In that case, the fact that our minds run on similar substrates is probably non-coincidental.

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by akvadrako on Saturday May 12, 2018 @07:25AM (#17053538)

Indeed, the connections seem profound. It seems to be a general-purpose optimal algorithm for, well, optimisation. And that would explain why the universe, our brains and AIs all trend toward it.

It could also be just that intelligence tends to mirror the outside world, but that seems a bit arbitrary.

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by vanderZwan on Saturday May 12, 2018 @10:02AM (#17054108)

> It could also be just that intelligence tends to mirror the outside world, but that seems a bit arbitrary.

We are part of the universe, so why would it be arbitrary if our brains were structured in ways that match typical structures found in this universe?

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by sitkack on Sunday May 13, 2018 @12:38PM (#17059760)

https://en.wikipedia.org/wiki/Panpsychism

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by vanderZwan on Sunday May 13, 2018 @05:10PM (#17061066)

I was thinking more along the lines that if our minds process outside information in a way that makes sense of that information, partially through simulating it, it does not seem so strange if the structures end up matching the outside structures through some form of convergent evolution.

From a cursory reading of that article I do not see it argue the same thing.

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by sitkack on Monday May 14, 2018 @03:40PM (#17068745)

That the universe is the best simulator of itself? That say, simulating water flowing through a pipe, the system doing the simulation re-formulates itself into something that resembles the pipe and the water?

[ Parent | on HN ]

Re:Why does deep and cheap learning work so well? by vanderZwan on Wednesday May 16, 2018 @10:30AM (#17082669)

> That the universe is the best simulator of itself?

What is this even supposed to mean? Also, "pipe" and "water" are ridiculously high level constructs, categorisations made by humans. Neither says anything about structures inherent to the universe.

I mean that when working with symmetries, information flow, and fundamental building blocks, certain structures just tend to pop up naturally. Hence fractals and geometrically shapes in places where you might not expect them. Or how laws of thermodynamics suddenly seems to be everywhere in biology all of a sudden now that we started looking[0][1].

[0] http://ecologicalsociology.blogspot.de/2010/11/geoffrey-west...

[1] https://www.quantamagazine.org/a-thermodynamic-answer-to-why...

[ Parent | on HN ]