Short answer no.

Long answer, the input you are thinking of (particle interactions) is already what partly defines the theory you are trying to uncover. So you have a serious chicken-egg situation.

You could think about inputting the data that comes out of particle detectors to look for patterns etc. and that sort of thing is being done already. However there is a difference between finding existing patterns and then figuring out the underlying theory which is the mathematical construct that lets you predict things.

Finally, even if you could have a "black box" predict stuff (correctly), the black box-ness is a serious problem for scientists from a meta-science or philosophy of science point of view, and people would be highly unsatisfied until they actually understood what was going on.

> Machine learning can observe and learn patterns that are more complex than humans can grasp.

That is a common misconception. ML cannot do anything beyond our modeling ability because it is designed with it. Deep learning is simply a method to approximate a function with a nonlinear formula. Something that cannot be easily approximated this way may require too much memory and power to be practical.

It is fundamentally similar to how JPEG is not a good fit for storing text: glyphs are hard to approximate with Fourier transforms.

The edge that ML has against humans is not in the learning part, it is in the machine part. Human memory is volatile, while we have grown exceedingly good at making machines retain memory.

That is not a valid argument. You need to provide a reason why the patterns that machine learning grasps are all graspable by humans, or that humans grasp something that machine learning never will. Multilayer neural networks can capture very interesting (from a human perspective) patterns and concepts, but also many others that seem garbage to us (perhaps because we dont grasp their significance).

Give me a ML system, and I can give you a problem it cannot solve. I am guaranteed success thanks to the No Free Lunch theorem: https://en.wikipedia.org/wiki/No_free_lunch_theorem.

In the case of deep learning, I can point to the task of determining values above 0.5 on an infinite Perlin-noise-derived 2D space fed by Mersenne Twister with seed 0, with an infinite number of octaves. Deep learning does not deal well with infinite spaces to begin with, and the pseudo-random generator cannot be easily encoded with common neural network nonlinear functions.

On the other hand, while we cannot compute an infinite number of octaves, and while places extremely far or extremely small details will run into IEEE754 limitations, we will get a good approximation by writing the program that computes the texture.

And that is just with only two numbers as input and one number as output.

Sure, I choose... "Exhaustive Search in the space of programs". (maybe with some genetic algorithm heuristics to shave of a couple billion years on each query)

It's a ML system that can solve any decidable and even some semi decidable problems. Which is (if the church turing thesis holds) everything that can be understood by humans or other.

You might not be able to wait around long enough to see it give you a result though, but hey at least _it_ got an answer.

If you allow impractical ML systems, you might as well pick a dice. Sure, the answer is inaccurate, but there's a non-zero probability that it is correct!

But, realistically, the ML system you devise cannot learn about features that require knowledge outside of the observable universe.

How does a static dice model computational processes?

What does the universe have to do with the set of computable functions?

Humans are neither immune to No Free Lunch, nor able to predict Perlin noise.

They're definitely not immune to No Free Lunch, but they are able to predict Perlin noise.

(Sure, you could try to analyse humans as if they were spherical objects floating in void, but in practice humans have computers.)

Let me give you an example. The 2011 Nobel prize in chemistry was dedicated to the discovery and analysis of quasicrystals. Those also cannot be modeled by deep learning, as its building blocks, linear separators, cannot finitely appreciate infinitely generated structures (unless it essentially encodes a completely different ML system within its neural network). Yet humans can model them.

I could go on all day about this, as there is an infinity of problems where deep learning is inadequate: proving the three-color theorem, routing, computing multiplications, …

Don't get me wrong: deep learning is outstanding for a set of menial tasks that I love to see being handed off to machines. But it is not the be-all, end-all that is sometimes claimed.

In principle its not impossible.

> Deep learning is simply a method to approximate a function with a nonlinear formula.

I think that was single hidden layer neural networks. Deep learning improves behaviour of that approximated function for values that were not in training set beyond what was possible with single layer (which was totally sufficient to aproximate any function).

Deep doen't improve ability to approximate function results but it it improves ability to approximate function implementation to get better result on the data that NN was not trained for.

I wonder, if given a big set of time1, time2 pairs (with relatively short dt) of the positions of the planets in our solar system, as seen in the night sky on earth, a machine learning system would be able to come up with good way to predict where the planets are going, given their positions.

Surely a machine learning system would need to be able to solve this 18th century physics problem, before we throw modern physics problems at it.

Pretty sure what you will end up with in the best case is a numerical integrator of the solar system as represented by a neural net. I also expect to have pretty crappy stability, but its an interesting question what sort of time step method it would approximate.

Maybe advanced AI of that nature could be the way forward in physics. I was quite good at physics at school etc. but have quite a job mentally even figuring how a gyroscope works and my brain totally fuzzes over at attempts to combine quantum field theory with general relativity. It may be that the theory of everything is there but just a bit hard for our ape brains to process. It would probably require better AI than we have at the moment though.

A little closer to what you asked about, Demis Hassabis of Deep Mind has kind of suggested:

>I was giving a talk at CERN a few months ago; obviously they create more data than pretty much anyone on the planet, and for all we know there could be new particles sitting on their massive hard drives somewhere and no-one’s got around to analyzing that because there’s just so much data. So I think it’d be cool if one day an AI was involved in finding a new particle.

So, how many bits are you giving the deep learning machine? If you give it enough bits, you aren't predicting anything -- you're just feeding it experimental data which it more or less spits back out.

For an elegant solution, one would need to have deep learning that also optimises on a constrained set of bits.

I feel like in this scenario, humans perform much better. The main strength of computer learning is being able to harness a massive dataset and really good storage capabilities.

Also, you would need to impose a "consistency" constraint on the computer which would be hard to do. Like a computer might say if (mass > 5) do this; else do that. And that is valid computationally. We do that in our split between gr/qm. But in some sense we feel this is wrong physically. The universe shouldn't run on arbitrary if statements.

So I think the answer is just no: the computer algorithms we have today can't handle the problem's constraints.

Enough bits to make it efficient at predicting. When it predicts better than the current theory, it starts getting closer to having enough bits. That's the beauty of ML, you don't need to worry about these details if it gives good accuracy.

My intuition was that there could be different ways to explain the laws of Physics that don't look like the current ones which evolved based on human intuition, math and language ability. A non-anthropocentric Physics if you will.

>That's the beauty of ML, you don't need to worry about these details if it gives good accuracy.

I get very alarmed by this. At work we have several examples of ML systems that have done good things for many years before suddenly and inexplicably blowing up and producing nonsense. Our folk explanation (as we have failed to produce anything resembling a proper one) is that the models that are captured in some cases appear to replicate reality but are deficient of some fundamental part of it which later comes into play destroying there predictive power. The domain theory changes in a sense, in another sense the driver was there all along but just hadn't featured in this part of the regime.

In ML, they might say you were overfitting. Predicting is all well and good, but predicting too well can indicate the machine hasn't really learned anything. It's just spitting back nearly identical information as the original.

It sounds like that is what this black box is supposed to do. All you need for a perfect black box is every possible data point...

So we have instance based learning which basically is about approximating this, and support vector machines which abstract the idea into representing the space of decisions represented by all the data points via a kernel which is a mapping function. The problem I see which deep learning seems to be "winning on" is that the space of instances that you have does not either abstract a meaningful theory of the distribution of these points (therefor predicting outside of this space) or describe things exterior to that space - but that exist, or might exist. A scientific theory is considered valid if it predicts things that have not been seen yet (and you can then test that, hence the post today about "string theory : enough"), I think IBL and SVMs can't represent the unseen, I used to work using something called inductive logic programming (I used a tool called Progol that Steven Muggleton made, it was good!) and I thought that that did produce such insights, sometimes, but it turned out that often these were actually me due to fiddling. Some play projects I did recently made me think that conv. nets were doing the same sort of things, but it was /is harder for me to catch and show this (and as I say, I ended up thinking that I'd often fooled myself with ILP, even though it was very useful).

The old skool difficulty I have with deep nets is that I used to think in terms of structural risk minimization vs empirical risk minimisation when dealing with overfitting. The idea was that if you had the right size of information store in your learned system and it was experimentally producing results that showed it was an effective predictor you could say that it was generalizing properly. Deep nets seem to me to have all the information storing capability of the domains they address and I worry....

But I am shocked, shocked, by how well they seem to work.

The idea that the universe runs on a simple program with simple rules hasn't failed us so far. All we have to do is figure out what that simple program is with our capability for abstract symbolic thinking and reasoning; machines are presently very bad at this while humans are less bad at it.

>The idea that the universe runs on a simple program with simple rules hasn't failed us so far.

Uhhhh... yes it has. The universe has myriad levels at which a simple program works. If the universe "really" runs on a simple program with simple rules, it's got so large a parameter space that even knowing the program itself is mostly useless to us.

https://arxiv.org/abs/1303.6738

A simple program running for billions years on trillions of bits...of course the result isn't going to be "simple." Still, knowing what the code would hardly be useless, and we are spending a lot of resources on trying to figure out.

Look up automated proofs. This exists. It's largely useless, due to mathematical properties of logic. Every mathematical system has things which are true but can't be discovered from first principals.

> Every mathematical system has things which are true but can't be discovered from first principals.

This is not true, see Gödel's completeness theorem [1], which states "... that a deductive system of first-order predicate calculus is 'complete' in the sense that no additional inference rules are required to prove all the logically valid formulas." It basically means that every "true" statements are provable with a good enough deductive system.

What you possibly think about is Gödel's first incompleteness theorem [2] which is quite a different statement. It states that in "most" formal systems (that are complex enough to be able to describe natural numbers) there is always a statement A that both A and "not A" are not provable. It basically means that these statements can't be "logically valid", which means that there exist models of the formal system where A is true and models where "not A" is true.

[1] https://en.wikipedia.org/wiki/G%C3%B6del%27s_completeness_th... [2] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

Nah, P vs NP is a bigger factor here.

Basically, checking a proof for correctness takes polynomial time in the size of the proof. (Dependending on how you formalize your proofs, that might even be linear.)

Coming up with a reasonable sized proof is much harder. But if we had P=NP, then checking and finding a proof would be about equally hard.

See eg http://www.scottaaronson.com/papers/philos.pdf

What would be the advantage of that over doing zillions of particle interactions in an actual particle accelerator?

If history is any guide then theories of physics are extremely simple. All the way from classical mechanics to special relativity, general relativity, quantum mechanics, quantum field theory, the theories are simple. In fact in some sense our theories have been getting simpler.

Take for instance the step from classical mechanics to special relativity. In classical mechanics time and space are not on equal footing. Time is the independent variable, and the space coordinates of particles are functions of time. In special relativity these are put on equal footing, and this simplifies the physics. It's not easier, but it's simpler in the sense that there are fewer parts. You used to have energy, and x-momentum, y-momentum, z-momentum for the 3 spatial directions. It turned out that energy is t-momentum in the time direction. This simplifies the equations because instead of having one equation for energy and another for x,y,z-momentum, you have one equation for x,y,z,t-momentum. Symmetries between electric and magnetic fields in Maxwell's equations were discovered that unified them into one object, just like with energy and momentum. A similar story exists for quantum mechanics.

What you suggest about modeling physics as a black box has been suggested before. After the first particle accelerators were built people were finding lots and lots of new particles, and there wasn't any good theory for them. Some people thought that we had to give up on a theory, and just model the real world as a black box of X particles in -> Y particles out, and make a big database of such interactions. However, then the standard model came along and all those particles turned out to consist of a much smaller number of quarks that interact in a small number of definite ways.

With string theory however we are in precisely the opposite situation. We don't have a big amount of experimental data that we need to find a theory for. We have a large number of theories and no experimental data to distinguish them.

I don't think so.

I wish the Frame Problem[1] got talked about a little more, because it's one of the fundamental challenges for intelligent beings/systems. I think we'd see fewer grandiose claims about AI if people spent a little more time pondering it. But I'm getting ahead of myself.

You can think of the Frame Problem as the problem babies have when they first enter the world. You've got eyes that can look anywhere, ears that are hearing sounds at every frequency, every touch receptor in your body is feeling something... There is data coming in from every pore and it's all noise.

Where to start? You can randomly twitch a muscle, but there are a lot of muscles. The likelihood that anything coherent will happen is very small. You could just pick sensors at random and try to correlate them with each other, and that lets you pick up some regularity, which would be useful for perception, except that regularity is still totally valueless. You can figure out how to see lines moving across a visual field, but there's no way to know if they are good or bad. Or what you might want to do with them. There is a sense in which it doesn't matter how much computing machinery you have, it's impossible to learn to perceive the world without outside help.

That's the frame problem.

Now, if you are a human, you get bootstrapped into the world through social interaction. You have very dumb insect-like circuits in your nervous system that make black dots with white on either side (i.e. another human eye) look very enticing to you. Before you've had any indication about whether these other eyes are a good thing or a bad thing, your muscles will twitch and you will (clumsily) orient all of your sensors (which are just screeching noise at you) towards whatever the other eyeballs around you are orienting towards. There are a handful of buttons on your body that create pleasure (warmth on your skin) and pain (pressure on your skin). That combined with some places to look gets you started. A human being builds from there, but we continue to get a ton of help throughout our whole life.

Machine learning is the same way. You can train a network to pick a dog out of a bunch of pictures of cats, but you have to give it know the difference between dogs and cats first, so you can feed the network a training set that it can learn from. If you just fed the network the pictures, without categorizing them, all it would see would be noise.

Back to physics... You can feed as much data as you want to a network, "deep" or not, and it will learn nothing. You need to be able to coherently structure it first. In chess that's easy: wins and losses. In physics, not so much. What's a win in physics? Explaining a situation that a bunch of other theories can't explain. So you'd have to structure all of the existing theories in physics in some machine-readable way. And then you'd have to somehow generate the space of all possible theories and all possible experiments....

And that still only gets you basically to where the baby was when they only had some pain and pleasure receptors. I.e. you are still totally screwed by the frame problem. Because you have no idea where to start looking through all of those possible experiments and all of those possible theories.

[1] https://en.wikipedia.org/wiki/Frame_problem

That's been solved by the most basic approach to ML: Supervised learning. A simplistic way to describe it would be that you say to the ML algorithm: "Here's two states, one at time N and one at time N+1. Your task is to predict the state at N+1 given the state at N. We have a bazillion of such state-pairs that you can test and refine your model on. Go nuts."

Supervised learning is what I'm talking about and it's not a solution to the frame problem. The human handler knows roughly the answers they looking for, and the machine learns details. The machine isn't discovering a new idea, it's just learning how to perform well inside a very small, well defined game that a human made.

It actually kind of has been done, can't find a link now...

I was actually thinking about this earlier in the day. Could we derive the law of universal gravitation if all we had was a bunch of data about the positions of objects at times plus a bunch of machine learning algorithms, and were otherwise complete idiots?

But in this case, no, because there's nothing in the particle data that we have that can't already be explained by the standard model, plus any number of other possible models. If only we could find something in the data that we couldn't explain, we might be able to make some progress, but that data doesn't exist.

possibly but the answer might be 42