And sure, it doesn't output a nice simple equation. But in most real problems, there aren't nice simple equations to output. Physics is sort of a fluke, where the fundamental laws of nature happen to be so simple we can do that. In something like biology, you are never going to find a simple equation that explains a biological system. They are complicated messes of millions of interacting pieces. Even if we knew how everything worked, we don't have the computing resources to accurately simulate it anyway.
And we are actually seeing neural networks be useful in these areas. They are able to predict what chemicals are more likely to be useful drugs. If you restricted yourself to using only simple mathematical models, you would never get these results.
This very article is about the very difficult domain of image processing. Why should we expect the statistical distribution of photographs to be simple and explainable by simple math? Finding an equation that accurately models the shape of a human face is pointless and misguided. Before NNs came along, they were doing crazy stuff like handcoding complicated heuristics that detected dark spots that might be eye sockets. There's nothing elegant about that. Physicists are really blessed to have a domain where they can ignore stuff like air resistance. Where the data really does fit simple equations.
And as the parent comment says, you can interpret neural networks. You can observe the activation behaviors, and discover what features it's learned to detect eye sockets, or chemicals with certain properties. You can see how the output changes as the inputs change, and fit a simpler model to that.
If you are simply amassing data without purpose then what you could do is to try to visualize the layers of the network to see if any surprising features turn up but that will only work if the features are obvious enough to stand out and if that were the case I would suspect we would not have this discussion in the first place.
Computers excel at: remembering stuff forever and speed.
So any kind of improvement that a neural net or any other solution would bring to the table over a human would likely fall in either one of those categories, either the computer is faster at solving the problem, or its ability to remember and apply a vast amount of data to the problem will give it a (slight) edge over what a human could do, or maybe it will be just enough to reach parity.
It will not tell you what is and what is not significant in the input, though, with enough samples you with your slower but much superior intellect might be able to draw new and far-reaching conclusions once you are exposed to the data in sufficient quantity yourself that a pattern suggests itself.
That's in a way a very nice collaboration between man and machine, each doing what they are best at.
A key element in something like this being at play would be the computer consistently being at odds with experts in the field but being right more often than not about those cases. That would be an excellent opportunity to wake up to the possibility that there is something that should be obvious and noticeable but that still got missed.
> So any kind of improvement that a neural net or any other solution would bring to the table over a human would likely fall in either one of those categories, either the computer is faster at solving the problem, or its ability to remember and apply a vast amount of data to the problem will give it a (slight) edge over what a human could do, or maybe it will be just enough to reach parity.
> It will not tell you what is and what is not significant in the input, though, with enough samples you with your slower but much superior intellect might be able to draw new and far-reaching conclusions once you are exposed to the data in sufficient quantity yourself that a pattern suggests itself.
I think that its too early to say how AI will be deployed in practice -- whether it will augment or replace human roles. The ambition is certainly to produce enough "intelligence" to supersede a large fraction of human decision making.
As we seek to make computers more powerful, robust, and versatile ("AI"), it seems like we are pushed towards more organic computational structures. If the trend continues, it would imply that the closer we get to AI, the less their strengths and weaknesses would resemble computers of yesteryear. The interesting possibility is that one might be able to have an interpolation between the strengths of humans and computers.
That means one should expect them to perform tasks about as well as a mouse brain can do them, between a third and halfway to adulthood.
And yes, that level of cortical processing does not seem to support coming up with symbolic systems and attempting to use them to describe the world around them. Absolutely.
Mice don't do that either.
It is not impossible. There is a combination of a special neural network architecture and strong sparsity-inducing regularization that makes it possible to learn equations from dynamics dataset: https://openreview.net/pdf?id=BkgRp0FYe
I'm not sure if there's been work done in the domain of pattern recognition, but here's an overview where in an example, Navier-Stokes is identified from data:
https://sinews.siam.org/Details-Page/data-driven-discovery-o...
But this is really an optimization problem: representing your formula as a bunch of weights that are then used to drive a cascade of multiply-add rules to derive some output is an imprecise but roughly accurate way to model a problem. That there is a much more direct and analytical way of modeling that problem requires intelligence of a different order.
But a program like mathematica gets awfully close to that, so that's not a problem that is a good fit for being solved with a neural network even if it would work.
I'd prefer to use a neural network for problems that have a less well defined solution.
Come on, you can just fit the NN numerical outputs with Excel and come up with tentative explanations for both the underlying elegant rule and the science. The point with NN-based science being you are not left guessing much, instead retrofit the NN numerical output and build a coherent narrative with other accepted results.
On the other hand...
Maybe a second NN could find that for you. You might end up with factors like "speed of an unladen swallow", but perhaps all that is needed is some nudge away from that local maxima to keep searching the space.
If a powerful idea is not elegant, we have the option of creating new mathematics in which it _can_ be elegantly expressed.
I think you are right on that one. The Standard Model for example is absoluteley not elegant, it is a huge convoluted mess of parameters and constants (just google for "Standard Model Lagrangian"). Yet, it is the best we came up with, explaining the dynamics of fields and particles[0], that surround us. Correct answers don't have to be elegant per se, its nice when they are, but it's not a prerequisite.
But, consider this: You can train a Neural Network to predict the distance of an object, thrown with velocity v_0, an angle of α, under the influcence of gravitational acceleration g. After a few hunderd rounds of training, a suited NN can reasonably predict the outcome of said experiment, with the input: v_0, α and g. And, as you pointed out, a researcher can explain to you why this is the case based on activation, feedback-loops, learning algorithm and other parameters. But neither the NN nor the researcher will be able to give you a rule like: "F=(G m_1 m_2)/(r^2)" to explain the underlying reasons for the object-trajectory dynamics.
A Neural Network can give you answers and predictions, yes, but you are not able to incorporate them into a wider theory, since the output is always numerical in nature. It is also always tied to one specific fact you are interested in and cannot give you a generalization.
[0]: In the model of QFT particles are also fields of a different kind.