Re:Deep distilling: Automated discovery of algorithms from data
by HanClinto on Wednesday February 21, 2024 @02:00PM
(#39457977)
My amateur take on this:
This paper makes me think of the underlying premise of perceptrons -- the universal approximation theorem -- which states that a neural network of simple perceptrons can be combined to approximate most any mathematical equation with reasonable accuracy. At least, I hope that's a reasonable approximation of what it says. :)
Even though real-life biological neurons are much more complex (and some can perform complex operations like XOR operations in a single node), a regular perception cannot produce XOR by itself.
But combine a bunch of perceptrons together across a few layers, and you can not only perform XOR with a neural network, but pretty much any mathematical equation.
This paper makes me think that they're attempting to go the reverse direction with it -- after training a neural network to model a quadratic formula or something -- can we then go backwards and approximate the neural network's behavior with a discrete math equation?
This is a fascinating idea, and if it works, would send us a LONG way towards building interpretable AI!
The repo links to the authors' preprint from 11/2021: https://arxiv.org/abs/2111.08275
https://rdcu.be/dy2Go
You can tell everyone "who is anyone" is in hysterics when this gets virtually no attention at Salamandar's old stomping ground here at ycombinator.
This paper makes me think of the underlying premise of perceptrons -- the universal approximation theorem -- which states that a neural network of simple perceptrons can be combined to approximate most any mathematical equation with reasonable accuracy. At least, I hope that's a reasonable approximation of what it says. :)
Even though real-life biological neurons are much more complex (and some can perform complex operations like XOR operations in a single node), a regular perception cannot produce XOR by itself.
But combine a bunch of perceptrons together across a few layers, and you can not only perform XOR with a neural network, but pretty much any mathematical equation.
This paper makes me think that they're attempting to go the reverse direction with it -- after training a neural network to model a quadratic formula or something -- can we then go backwards and approximate the neural network's behavior with a discrete math equation?
This is a fascinating idea, and if it works, would send us a LONG way towards building interpretable AI!