Maybe there’ll be an intermediate era for a while where ML models outperform traditional analytical science, but then eventually we’ll still be able to find the (hopefully limited in number) principles from which it can all be derived. I don’t think we’ll ever find that Occam’s razor is no use to us.
At that point I wonder if it would be possible to feed that uninterpretable model back into another model that makes sense of it all and outputs sets of equations that humans could understand.
OTOH, Newtonian mechanics is great at predicting things under certain circumstances yet, in the same way, doesn't necessarily reflect the underlying mechanism of the system.
So maybe philosophers will eventually tell us the distinction we are trying to draw, although intuitive, isn't real
That's very common. It's the reason to test the new drug in petri dish, then rats, then dogs, then humans and if all test passed send it to the pharmacy.
Perhaps the opportunity here is to provide a quicker feedback loop for theory about predictions in the real world. Almost like unit tests.
Or jumping the gap entirely to move towards more self-driven reinforcement learning.
Could one structure the training setup to be able to design its own experiments, make predictions, collect data, compare results, and adjust weights...? If that loop could be closed, then it feels like that would be a very powerful jump indeed.
In the area of LLMs, the SPAG paper from last week was very interesting on this topic, and I'm very interested in seeing how this can be expanded to other areas:
If you're a scientist who accepts that probabilist models beat interpretable ones (articulated well here: https://norvig.com/chomsky.html), then you'll be quite happy because this is yet another validation of the value of statistical approaches in moving our ability to predict the universe forward.
If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail, you'll find this an interesting challenge- can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models?
If you're the sort of person who likes simple models with as few parameters as possible, you're probably excited because developing more comprehensible or interpretable models that have equivalent predictive ability is a very attractive research subject.
(FWIW, I'm in the camp of "we should simultaneously seek simpler, more interpretable models, while also seeking to improve native human intelligence using computational augmentation")
AlphaFold 3 can rapidly reduce a vast search space in a way physically-based methods alone cannot. This narrowly focused search space allows scientists to apply their rigorous, explainable, physical methods, which are slow and expensive, to a small set of promising alternatives. This accelerates drug discovery and uncovers insights that would otherwise be too costly or time-consuming.
The future of science isn't about AI versus traditional methods, but about their intelligent integration.
My only worry is that AlphaFold and others, e.g. ESM, seem to be bit fragile for out-of-distribution sequences. They are not doing a great job with unusual sequences, at least in my experience. But hopefully they will improve and provide better uncertainty measures.
It’s actually required as part of the submission for FDA approval that you posit a specific Mechanism of Action for why your drug works the way it does. You can’t get approval without it
Vioxx is a nice example of a molecule that got all the way to large-scale deployment before being taken off the market for side effects that were known. Only a decade before that, I saw a very proud pharma scientist explaining their "mechanism of action" for vioxx, which was completely wrong.
Some had tried to come up with other criteria to confirm you have discovered an underlying principle without predictive power, such as on aesthetics - but this is seen by the majority of scientists as basically a cop out. See debate around string theory.
Note that this comment is summarizing a massive debate in the philosophy of science.
for many basic/fundamental mathematical objects we don't (yet) have simple mechanistic ways to compute them.
so if a probabilistic model spits out something very useful, we can slap a nice label on it and call it a day. that's how engineering works anyway. and then hopefully someday someone will be able to derive that result from "first principles" .. maybe it'll be even more funky/crazy/interesting ... just like mathematics arguably became more exciting by the fact that someone noticed that many things are not provable/constructable without an explicit Axiom of Choice.
https://en.wikipedia.org/wiki/Nonelementary_integral#Example...
Yes, but we're taking about roughly the opposite of a proof
and it seems with these molecular biology problems we constantly have the problem of specificity (model prediction quality) vs sensitivity (model applicability), right? but due to information theory constraints there's also a dimension along model size/complexity.
so if a ML model can push the ROC curve toward the magic left-up corner then likely it's getting more and more complex.
and at one point we simply are left with models that are completely parametrized by data and there's virtually zero (direct) influence of the first principles. (I mean that at one point as we get more data even to do model selection we can't use "first principles" because what we know through that is already incorporated into previous versions of the models. Ie. the information we gained from those principles we already used to make decisions in earlier iterations.)
Of course then in theory we can do model distillation, and if there's some hidden small/elegant theory we can probably find it. (Which would be like a proof through contradiction, because it would mean that we found model with the same predictive power but with smaller complexity than expected.)
// NB: it's 01:30 here, but independent of ignorance-o-clock ... it's quite possible I'm totally wrong about this, happy to read any criticism/replies
You’ve discovered magic.
When you read about a wizard using magic to lay waste to invading armies, how much value would you guess the armies place in whether or not the wizard truly understands the magic being used against them?
Probably none. Because the fact that the wizard doesn’t fully understand why magic works does not prevent the wizard from using it to hand invaders their asses. Science is very much the same - our own wizards used medicine that they did not understand to destroy invading hordes of bacteria.
There is a car. We think it drives by burning petrol somehow.
How do we test this? We take petrol away and it stops driving.
Ok, so we know it has something to do with petrol. How does it burning the petrol make it drive?
We think it is caused by the burned petrol pushing the cylinders, which are attached to the wheels through some gearing. How do we test it? Take away the gearing and see if it drives.
Anyway, this never ends. You can keep asking questions, and as long as the hypothesis is something you can test, you are doing science.
You discovered a principle.
Better example:
There is a car. We don’t know how it drives. We turn the blinkers on and off. It still drives. Driving is useful. I drive it to the store
The best part is where the geneticist ties the arms of all the suit-wearing employees and it has no functional effect on the car.
Yes, but a perfect oracle has no explanatory power, only predictive.
That doesn’t diminish the value that patients received in any way even though it would be more satisfying to make predictions and design something to interact in a way that exactly matches your theory.
Newton was a bit of a brat but everybody accepted his explanation. Then the problem turned to trying to explain gravity.
Thus science advances, one explanation at a time.
Scientific method can help us rule out what underlying principles are definitely not. Any such principles are not actually up to be “discovered”.
If probabilistic ML comes along and does a decent job at predicting things, we should keep in mind that those predictions are made not in context of absolute truth, but in context of theories and models we have previously developed. I.e., it’s not just that it can predict how molecules interact, but that the entire concept of molecules is an artifact of just some model we (humans) came up with previously—a model which, per above, is probably incomplete/incorrect. (We could or should use this prediction to improve our model or come up with a better one, though.)
Even if a future ML product could be creative enough to actually come up with and iterate on models all on its own from first principles, it would not be able to give us the answer to the question of underlying principles for the above-mentioned reasons. It could merely suggest us another incomplete/incorrect model; to believe otherwise would be to ascribe it qualities more fit for religion than science.
People clearly have been able to discover many underlying principles using the scientific method. Then they have been able to explain and predict many complex phenomena using the discovered principles, and create even more complex phenomena based on that. Complex phenomena such as the technology we are using for this discussion.
Words dont have any inherent meaning, just the meaning they gain from usage. The entire concept of truth is an artifact of just some model (language) we came up with previously—a model which, per above, is probably incomplete/incorrect. The kind of absolute truth you are talking about may make sense when discussing philosophy or religion. Then there is another idea of truth more appropriate for talking about the empirical world. Less absolute, less immutable, less certain, but more practical.
Exactly—except you are talking about it, too. When you say “discovering underlying principles”, you are implying the idea of absolute truth where there is none—the principles are not discovered, they are modeled, and that model is our fallible human construct. It’s a similar mistake as where you wrote “explain”: every model (there should always be more than one) provides a metaphor that 1) first and foremost, jives with our preexisting understanding of the world, and 2) offers a lossy map of some part of [directly inaccessible] reality from a particular angle—but not any sort of explanation with absolute truth in mind. Unless you treat scientific method as something akin to religion, which is a common fallacy and philosophical laziness, it does not possess any explanatory powers—and that is very much by design.
You are assigning meanings to words like "discovering", "principles", and "explain" that other people don't share. Particularly people doing science. Because these absolute philosophical meanings are impossible in the real world, they are also useless when discussing the reality. Reserving common words for impossible concepts would not make sense. It would only hinder communication.
Science is pretty much the same. We can often discover how the reality works, and use the discoveries to explain many things. Somehow that keeps happening all the time, even if we can never be fully sure about anything.
Again: scientific method does not explain. Religion or philosophy are about explaining. Scientific method is about experimentation and making testable predictions. What experiments we perform is determined by how we understand the world, and if there is any subsequent explanation about “how things really are” (a.k.a. “the underlying principles”) then it has nothing to do with scientific method which does not make such claims by design; that is untestable/unfalsifiable beliefs and a product of either philosophical or religious thinking.
Since you insist on using specific words “explain” and “discover”, rather than more conventionally used in science “predict” or “model”, it implies they mean something different to you. I have provided the meanings of “explain” and “discover” I am familiar with, as it applies to the discussion at hand (which is about the philosophy of scientific process, underlying principles and truths about objectively existing reality). If you refuse to identify the meanings you are using those words in, I take it that you concede whatever point you had.
What you call the scientific method is a philosophical construct that has little to do with actual science. And philosophers disagree on whether it's a good ideal for science. Given that it's neither a good description of science nor a universal ideal for science, I wouldn't focus too much on it when discussing science.
I can’t help thinking I’m talking to an LLM or a troll.
If you use a complex term that needs definition in a casual discussion, it’s most likely none of the words you use in the definition would themselves require definitions—and if this was to happen repeatedly, the conversation would halt long before we would be running out of words. It’s enough to avoid circularity within a couple of levels in good faith.
Anyway, I’m not sure whether we disagree or not or what exactly we are arguing about. My point is “ML making predictions is not a threat to us getting at underlying principles, because natural science (scientific method, predicting things) in general does lead us to any provable facts about those principles, and because ML would make predictions within an incorrect/incomplete model that we gave it.” In that, by “underlying principles” I mean some statements about objective reality. If we are on the same page here, we can continue discussion, otherwise let’s not.
Similarly, scientific method cannot discover the underlying principles or explain the nature. It can only rule out principles and explanations. Regardless, science seems to come up with principles and explanations all the time.
And that's because scientific method is not science. It's a theoretical model for a subset of (often ritualized) activities within science. Actual science is more than that. It can do things scientific method cannot, because it's less focused on philosophical ideas such as absolute truth or provable facts.
In my experience, scientific method is like a picture of an elephant drawn from a written description. Drawn by someone who has never seen the animal or a picture of it, and who has no idea what kind of an animal it is. There are some recognizable elements, but it definitely does not look like the real thing.
High level, I see a distinction between theory and practice, between an oracle predicting without explanation, and a well-thought out theory built on a partnership between theory and experiment over centuries, ex. gravity.
I have this feeling I can't shake that the knife you're using is too sharp, both in the specific example we're discussing, and in general.
In the specific example, folding, my understanding is we know how proteins fold & the mechanisms at work. It just takes an ungodly amount of time to compute and you'd still confirm with reality anyway. I might be completely wrong on that.
Given that, the proposal to "dedicate...engineer[s] towards finding ethical ways to improve...intelligence so that we can appreciate the underlying principles better" begs the question of if we're not appreciating the underlying principles.
It feels like a close cousin of physics theory/experimentalist debate pre-LHC, circa 2006: the experimentalists wanted more focus on building colliders or new experimental methods, and at the extremes, thought string theory was a complete was of time.
Which was working towards appreciating the underlying principles?
I don't really know. I'm not sure there's a strong divide between the work of recording reality and explaining it. I'll peer into a microscope in the afternoon, and take a shower in the evening, and all of a sudden, free associating gives me a more high-minded explanation for what I saw.
I'm not sure a distinction exists for protein folding, yes, I'm virtually certain this distinction does not exist in reality, only in extremely stilted examples (i.e. a very successful oracle at Delphi)
Not saying ML methods haven't shown important reproducibility challenges, but to just shut them down due to not being "useful science" is inflexible.
This has happened before. Newtonian mechanics was incomprehensible spooky action at a distance, but Einstein clarified gravity as the bending of spacetime.
Like, quantum mechanics doesn’t seem, to me, to just be a way of describing how to predict things. I view it as saying substantial things about how things are.
Sure, there are different interpretations of it, which make the same predictions, but, these different interpretations have a lot in common in terms of what they say about “how the world really is” - specifically, they have in common the parts that are just part of quantum mechanics.
The qau that can be spoken in plain language without getting into the mathematics, is not the eternal qau, or whatever.
Can we differentiate?
Prediction is understanding. What we call "understanding" is a cognitive illusion, generated by plausible but brittle abstractions. A statistically robust prediction is an explanation in itself; an explanation without predictive power explains nothing at all. Feeling like something makes sense is immeasurably inferior to being able to make accurate predictions.
Scientists are at the dawn of what chess players experienced in the 90s. Humans are just too stupid to say anything meaningful about chess. All of the grand theories we developed over centuries are just dumb heuristics that are grossly outmatched by an old smartphone running Stockfish. Maybe the computer understands chess, maybe it doesn't, but we humans certainly don't and we've made our peace with the fact that we never will. Moore's law does not apply to thinking meat.
Now, if we look at history of science and technology, there is a shit ton of practical stuff that was found only by pure accident - discoveries of which could not be predicted from any previous theory.
I would view both A) and B) as net positives. But our teaching of the next generation of scientists needs to adapt.
The worst case scenario is of course that the middle management driven enshittification of science will proceed to a point where there are only few people who actually are scientists and not glorified accountants. But I’m optimistic this will actually super charge science.
With good luck we will get rid of the both of the biggest pathologies in modern science - 1. number of papers published and referred as a KPI 2. Hype driven super politicized funding where you can focus only one topic “because that’s what’s hot” (i.e. string theory).
The best possible outcome is we get excitement and creativity back into science. Plus level up our tech level in this century to something totally unforeseen (singularity? That’s just a word for “we don’t know what’s gonna happen” - not a specific concrete forecasted scenario).
It's more specific than you make it out. The singularity idea is that smart AIs working on improving AI will produce smarter AIs, leading to an ever increasing curve that at some point hits a mathematical singularity.
Nobody knows what singularity would actually mean from the point of view of specific technological development.
All our theories are built on observation, so these empirical models yielding such useful results is a great thing - it satisfies the need for observing and acting. Missing explainability of the models merely means we have less ability to act more precisely - but it does not devalue our ability to act coarsely.
The concern is about "holding it all in your head", and depending on your preferred level of abstraction, "all" can perfectly reasonably be held in your head. For example: "This program generates the most likely outputs" makes perfect sense to me, even if I don't understand some of the code. I understand the system. Programmers went through this decades ago. Physicists had to do it too. Now, chemists I suppose.
While computer operations in solutions are computable by humans, the billions of rapid computations are unachievable by humans. In just a few seconds, a computer can perform more basic arithmetic operations than a human could in a lifetime.
That the tools became complex is not a reason to fret in science. No more than statistical physics or quantum mechanics or CNN for image processing - it's complex and opaque and hard to explain but perfectly reproduceable. "It works better than my intuition" is a level of sophistication that most methods are probably doomed to achieve.
"This program generates the most likely outputs" isn't a scientific explanation, it's teleology.
Speech recognition. OCR. Reccomendation engines.
You don't write OCR by going "if there's a line at this angle going for this long and it crosses another line at this angle then it's an A".
There's too many variables and influence of each of them is too small and too tightly coupled with others to be able to abstract it into something that is understandeable to a human brain.
Or, consider the art word broadly, artists routinely engage in various forms of unusual abstraction.
It's unabstractable for people, because the most abstract model that works still has far too many variables for our puny brains.
> artists routinely engage in various forms of unusual abstraction
Abstraction in art is just another, unrelated meaning of the word. Like execution of a program vs execution of a person. You could argue executing the journalist for his opinions isn't bad, because execution of mspaint.exe is perfectly fine, but it won't get you far :)
Abstraction doesn't have to be perfect, just as "logic" doesn't have to be.
> Abstraction in art is just another, unrelated meaning of the word.
Speaking of art: have you seen the movie The Matrix? It's rather relevant here.
The value of pi is a simple counterexample.
If anyone actually thought this way -- no one does -- they definitely wouldn't build models like this.
It appears that your own comment is disproving this statement
I don't quite understand this point — could you elaborate?
My understanding is that the ML model produces a hypothesis, which can then be tested via normal scientific method (perform experiment, observe results).
If we have a magic oracle that says "try this, it will work", and then we try it, and it works, we still got something falsifiable out of it.
Or is your point that we won't necessarily have a coherent/elegant explanation for why it works?
Knowing how to predict the motion of planets but without having an underlying explanation encourages scientists to develop their theories. Now, once more, we know how to predict something (protein folding) but without an underlying explanation. Hurray, something to investigate!
(Aside: I realize that there are also more human factors at play, and upsetting the status quo will always cause some grief. I just wanted to provide a counterpoint that there is some exciting progress represented here, too).
Whether ML models produce hypotheses is something of an epistemiological argument that I think muddies the waters without bringing any light. I would only use the term "ML models generate predictions". In a sense, the model itself is the hypothesis, not any individual prediction.
Forget trying to understand dark matter. Just use this model to correct for how the universe works. What is actually wrong with our current model and if dark matter exists or not or something else is causing things doesn't matter. "Shut up and calculate" becomes "Shut up and do inference."
If you think carefully humans operate in the same regime. Our concepts are all like that - imperfect, approximative, glossing over some details. Our fundamental grounding and test is survival, an unforgiving filter, but lax enough to allow for anti-vaxxer movements during the pandemic - survival test is not testing for truth directly, only for ideas that fail to support life.
This seems to me an empirical question about the world. It’s clear our minds are limited, and we understand complex phenomena through abstraction. So either we discover we can continue converting advanced models to simpler abstractions we can understand, or that’s impossible. Either way, it’s something we’ll find out and will have to live with in the coming decades. If it turns out further abstractions aren’t possible, well, enlightenment thought had lasted long enough. It’s exciting to live at a time in humanity’s history when we enter a totally uncharted new paradigm.
I think we will have to develop a methodology and supporting toolset to be able to derive the underlying patterns driving such ML models. It's just too much for a human to comb through by themselves and make sense of.
Other Chomsky-like models of human grammars have different asymptotic behavior and different choices of n, but the same fundamental problem; the big-O constant factor isn't neurons firing but rather human connections between the n inputs. How can you conceive of human minds being able to track O(n^3) (or whatever) cost where that n is everything being communicated -- words, concepts, symbols, representations, all that jazz and the polynomial relationships between them?
But I feel an apology is in order: I've had quite a few beers before coming home, and it's probably a mistake to try to express academically charged and difficult views on the Internet while in an inebriated state. Probably the alcohol has substantially decreased my mental computational power. However, it has only mildly impaired my ability to string together words and sentences in a grammatically complex fashion. In fact, I often feel that the more sober and clear-minded I am, the simpler my language is. Maybe human grammar is actually sub-polynomial. I have observed the same in ChatGPT; the more flowery and wordy it has become over time, the dumber its output.
As an aside but relevant to your point, my entire introduction to DNA and protein analysis was based on Chomsky grammars. My undergrad thesis advisor David Haussler handed me a copy of an article by David Searls "The Linguistics of DNA" (https://www.scribd.com/document/461974005/The-Linguistics-of...) . At the time, Haussler was in the middle of applying HMMs and other probabilistic graphical models to sequence analysis, and I knew all about DNA as a molecule, but not how to analyze it.
Searls paper basically walks through Chomsky's hierarchy, and how to apply it, using linguistic techniques to "parse" DNA. It was mind-bending and mind-expanding for me (it takes me a long time to read papers, for example I think I read this paper over several months, learning to deal with parsing along the way). To this day I am astounded at how much those approaches (linguistics, parsing, and grammars) have evolved- and yet not much has changed! People were talking about generative models in the 90s (and earlier) in much the same way we treat LLMs today. While much of Chomsky's thinking on how to make real-world language models isn't particuarly relevant, we still are very deeply dependent on his ideas for grammar...
Anyway, back to your point. While CFGs may be O(n*3) I would say that there is a implicit, latent O(n) parseable grammar underlying human linguistics, and our brains can map that latent space to its own internal representation in O(1) time, where the n roughly correlates to the complexity of the idea being transferred. It does not seem even remotely surprising that we can make multi-language models that develop their own compact internal representation that is presumably equidistant from each source language.
I think chess engines, weirdly enough, have disabused me of this notion.
There are lots of factors a human considers when looking at a board. Piece activity. Bishop and knight imbalances. King safety. Open and semi-open file control. Tempo. And on and on.
But all of them are just convenient shortcuts that allow us to substitute reasonable guesses for what really matters: exhaustively calculating a winning line through to the end. “Positional play” is a model that only matters when you can’t calculate trillions of lines thirty moves deep, and it’s infinitely more important that a move survives your opponent’s best possible responses than it is to satisfy some cohesive higher level principle.
Ignore the existence of engines for a moment. The reason a particular line works, at the end of the day, is simply because it does. Just because we have heuristics that help us skip a lot of evaluation doesn’t mean the heuristics have intrinsic meaning within the game. They don’t.
They’re shortcuts that let us skip having to do the impossible. The heuristics will always lose to concrete analysis to a deep enough depth.
And that’s my point. We come up with models that give us an intuition for “why” things are a certain way. Those models are inarguably helpful toward having a gut feeling. But models aren’t the thing itself, and every model we’ve found breaks down at some deeper point. And maybe at some level things simply “are” some way with no convenient shorthand explanation.
So its clear that there is in fact 'deeper patterns to chess' that allow one to play very well, without any search required (Since LLMs cannot search). Its just that those patterns are probably rather different to human understood ones.
And at some level the answer is simply “because every possible refutation fails” and there is no simpler pattern to match against nor intuition to be had. That is the how and why of it.
Thank God! As a person who uses my brain, I think I can say, pretty definitively, that people are bad at understanding things.
If this actually pans out, it means we will have harnessed knowledge/truth as a fundamental force, like fire or electricity. The "black box" as a building block.
We've had stuff like this for a long time.
Notable examples:
- Temple priestesses
- Tea-leaf reading
- Water scrying
- Palmistry
- Clairvoyance
- Feng shui
- Astrology
The only difference is, the ML model is really quite good at it.
That's the crux of it: we've had theories of physics and chemistry since before writing was invented.
None of that mattered until we came upon the ones that actually work.
So in a way, what you say is already possible. Just how GMs in chess specialize in certain openings or play styles, master chemists have pre-existing biases that can affect their designs; algorithms can have different biases which push exploration to interesting places. Once you have a good latent representation of relevant chemical space, so you can optimize for this sort of creativity (a practical but boring example is to push generation outside of patent space).
For a lot of problems, currently you either don't have an an analytical solution and the alternative is a brute force-ish numerical approach. As a result the computational cost of simulating things enough times to be able to detect behavior that can inform theories/models (potentially yielding a good analytical result) is not viable.
In this regard, ML models are promising.
1. Research can then focus on where things go wrong
2. ML models, despite being "black boxes," can still have brute-force assessment performed of the parameter space over covered and uncovered areas by input information
3. We tend to assume parsimony (i.e Occam's razor) to give preference to simpler models when all else is equal. More complex black-box models exceeding in prediction let us know the actual causal pathway may be more complex than simple models allow. This is okay too. We'll get it figured out. Not everything is closed-form, especially considering quantum effects may cause statistical/expected outcomes instead of deterministic outcomes.
A better analogy is "weather forecasting".
Perhaps we're in an early stage of Ted Chiang's story "The Evolution of Human Science", where AIs have largely taken over scientific research and a field of "meta-science" developed where humans translate AI research into more human-interpretable artifacts.
I think it is fine & better for society to have applications and models for things we don't fully understand... We can model lots of small aspects of weather, and we have a lot of factors nailed down, but not necessarily all the interactions.. and not all of the factors. (Additional example for the same reason: Gravity)
Used responsibly. Of course. I wouldn't think an AI model designing an airplane that no engineers understand how it works is a good idea :-)
And presumably all of this is followed by people trying to understand the results (expanding potential research areas)
https://journals.aps.org/prresearch/abstract/10.1103/PhysRev...
I mean, it's just faster, no? I don't think anyone is claiming it's a more _accurate_ model of the universe.
World is still here, although the Matrix/metaverse is becoming more attractive daily.
Personally, I’m convinced that human reason is less pure than we think it to be, and that the move to large mathematical models might just be formalizing a lack-of-control that was always there. But that’s less of a philosophy of science discussion and more of a cognitive science one
I’ll take prediction over understanding if that’s the best our brains can do. We’ve evolved to deal with a few orders of magnitude around a meter and a second. Maybe dealing with light-years and femtometer/seconds is too much to ask.
Going back to the uninterpretable ML models in the context of AlphaFold 3, I think one method for trying to explain the findings is similar to the experimental methods of physics with reality: you perform experiments with the reality (in this case AlphaFold 3) to came up with sound conclusions. AI/ML is an interesting black-box system.
There are other open discussions on this topic. For example, can our human brain absorbe that knowledge or it is limited somehow with the scientific language that we have now?
[1] https://www.google.com.ar/books/edition/New_Directions_in_th...
Imagine if optis was complex enough that it would require ML model to predict anything.
We'd be in permanent stone age without a way out.
It makes me think of how Gaussian integers have irreducibles but not prime numbers, where some large things cannot be uniquely expressed as combination of smaller things.
We already know how to model the vast majority of things, just not at a speed and cost which makes it worthwhile. There are dimensions of value - one is accuracy, another speed, another cost, and in different domains additional dimensions. There are all kinds of models used in different disciplines which are empirical and not completely understood. Reducing things to the lowest level of physics and building up models from there has never been the only approach. Biology, geology, weather, materials all have models which have hacks in them, known simplifications, statistical approximations, so the result can be calculated. It's just about choosing the best hacks to get the best trade off of time/money/accuracy.
In reality, science has already pretty much gone this way long ago, even if people don't like to admit it. Simple, reductionist explanations for complex phenomena in living systems don't really exist. Virtually all of medicine nowadays is empirical: try something, and if you can prove its safe and effective, you keep doing it. We almost never have a meaningful explanation for how it really works, and when we think we do, it gets proven wrong repeatedly, while the treatment keeps working as always.
Imagine a very large room that has every surface covered by on-off switches.
We cannot see inside of this room. We cannot see the switches. We cannot fit inside of this room, but a toddler fits through the tiny opening leading into the room. The toddler cannot reach the switches, so we equip the toddler with a pole that can flip the switches. We train the toddler, as much as possible, to flip a switch using the pole.
Then, we send the toddler into the room and ask the toddler to flip the switch or switches we desire to be flipped, and then do tests on the wires coming out of the room to see if the switches were flipped correctly. We also devise some tests for other wires to see if that naughty toddler flipped other switches on or off.
We cannot see inside the room. We cannot monitor the toddler. We can't know what _exactly_ the toddler did inside the room.
That room is the human body. The toddler with a pole is a medication.
We can't see or know enough to determine what was activated or deactivated. We can invent tests to narrow the scope of what was done, but the tests can never be 100% accurate because we can't test for every effect possible.
We introduce chemicals then we hope-&-pray that the chemicals only turned on or off the things we wanted turned on or off. Craft some qualifications testing for proofs, and do a 'long-term' study to determine if there were other things turned on or off, or a short circuit occurred, or we broke something.
I sincerely hope that even without human understanding, our AI models can determine what switches are present, which ones are on and off, and how best to go about selecting for the correct result.
Right now, modern medicine is almost a complete crap-shoot. Hopefully modern AI utilities can remedy the gambling aspect of medicine discovery and use.
For example, we have a good idea of why certain antibiotics cure tuberculosis - we understand that tuberculosis is caused by certain bacteria, and we know how antibiotics affect the cellular chemistry of those bacteria to kill them. We also understand the dynamics of this, the fact that the body's immune system still has to be functioning well enough to kill many of the bacteria as well, etc. We don't fully understand all of the side-effects and possible interactions with other diseases or medications in every part of the body, but we understand the gist of it all.
Then there are drugs and diseases where we barely understand any of it. We don't have for example a clear understanding of what depression is, what the biochemistry of it is. We do know several classes of drugs that help with depression in certain individuals, but we know those drugs don't help with other individuals, and we have no way of predicting which is which. We know some of the biochemical effects of these drugs, but since we don't understand the underlying cause of depression, we don't actually know why the drugs help, or what's the difference in individuals where they don't help.
There are also widely used medications where we understand even less. Metamizole, a very widely used painkiller sold as Novalgin or Analgin and other names, discovered in 1922, has no firmly established mechanism of action.
That said, I don’t even know if ML is good at finding patterns in data.
That's the only thing ML does.
But yet the only thing that can save us from ML will be ML itself because it is ML that has the best chance to be able to extrapolate patterns from these blackbox models to develop human interpretable models. I hope we do dedicate explicit effort to this endeavor, and so continue the human advances and expanse of human knowledge in tandem with human ingenuity with computers at our assistance.
Open a random physics book, and you will find lots and lots of derivations (using more or less acceptable assumptions depending on circumstance under consideration).
Derivations and assumptions can be formally verified, see for example https://us.metamath.org
Ever more intelligent machine learning algorithms and data structures replacing human heuristic labor, will simply shift the expected minimum deliverable from associations to ever more rigorous proofs in terms of less and less assumptions.
Machine learning will ultimately be used as automated theorem provers, and their output will eventually be explainable by definition.
When do we classify an explanation as explanatory? When it succeeds in deriving a conclusion from acceptable assumptions without hand waving. Any hand waving would result in the "proof" not having passed formal verification.
Iirc when EU required banks to have interpretable rules for loans, a plain explanation was not considered enough. What was required was a clear process that was used from the beginning - i.e. you can use an AI to develop an algorightm to make a decision, but you can’t use AI to make a decision and explains reasons afterwards.
> One strong theme is the prevalence of context features (e.g. DNA, base64) and token-in-context features (e.g. the in mathematics – A/0/341, < in HTML – A/0/20). 29 These have been observed in prior work (context features e.g. [38, 49, 45] ; token-in-context features e.g. [38, 15] ; preceding observations [50] ), but the sheer volume of token-in-context features has been striking to us. For example, in A/4, there are over a hundred features which primarily respond to the token "the" in different contexts. 30 Often these features are connected by feature splitting (discussed in the next section), presenting as pure context features or token features in dictionaries with few learned features, but then splitting into token-in-context features as more features are learned.
> [...]
> The general the in mathematical prose feature (A/0/341) has highly generic mathematical tokens for its top positive logits (e.g. supporting the denominator, the remainder, the theorem), whereas the more finely split machine learning version (A/2/15021) has much more specific topical predictions (e.g. the dataset, the classifier). Likewise, our abstract algebra and topology feature (A/2/4878) supports the quotient and the subgroup, and the gravitation and field theory feature (A/2/2609) supports the gauge, the Lagrangian, and the spacetime
I don't think "hundreds of different ways to represent the word 'the', depending on the context" is a-priori plausible, in line with our preconceptions, or aesthetically pleasing. But it is what falls out of ML interpretation techniques, and it does do a quantitatively good job (as measured by fraction of log-likelihood loss recovered) as an explanation of what the examined model is doing.
It could lead to a very interesting ecosystem of roles.
Even if you just limit the discussion to using the best model of X to design a better Y, limited to the model's domain of validity, that might translate the usage problem to finding argmax_X of valueFunction of modelPrediction of design of X. In some sense a good predictive model is enough to solve this with brute force, but this still leaves room for tons of fascinating foundational work. Maybe you start to find that the (wow so small) errors in modelPrediction are correlated with valueFunction, so the most accurate predictions don't make it the best for argmax (aka optimization might exploit model errors rather than optimizing the real thing). Or maybe brute force just isn't computationally feasible, so you need to understand something deeper about the problem to simplify the optimization to make it cheap.
I can only assume that existing methods would still be used for verification. At least we understand the logic used behind these methods. The ML models might become more accurate on average but they could still throw out results that are way off occasionally, so their error rate would have to become equal to the existing methods.
In theoretical physics, you know the equations, you solve equations analytically, but you can only do that when the model is simple.
In numerical physics, you know the equations, you discretize the problem on a grid, and you solve the constraint defined by the equations with various numerical integration schemes like RK4, but you can only do that when the model is small and you know the equations, and you find a single solution.
Then you want the result faster, so you use mesh-free methods and adaptive grids. It works on bigger models but you have to know the equations, finding a single solution to the differential equations.
Then you compress this adaptive grid with a neural network, while still knowing the governing equations, and you have things like Physics Informed Neural Networks ( https://arxiv.org/pdf/1711.10561 and following papers) where you can bound the approximation error. This method allows solve all solutions to the differential equations simultaneously, sharing the computations.
Then when knowing explicitly your governing equations is too complex, so you assume that there are some governing stochastic equations implicitly, which you learn the end-result of the dynamic with a diffusion model, that's what this alpha-fold is doing.
ML is kind of a memoization technique, analog to hashlife in the game of life, that allows you reuse your past computational efforts. You are free to choose on this ladder which memory-compute trade-off you want to use to model the world.
It seems like we're repeating that here, albeit with wildly different methods. We're getting better models but by giving up on the possibility of actually understanding things from first principles.
So when you're tools can produce outputs that you find useful, you can then use those tools to develop your understanding and insights. As a tool, this is quite good.
what happens is an opportunity has entered the chat.
There is a wave coming—I won't try to predict if it's the next one—where the hot thing in AI/ML is going to be profoundly powerful tools for analyze other such tools and render them intelligible to us,
which will I imagine mean providing something like a zoomable explainer. At every level there are footnotes; if you want to understand why the simplified model is a simplification, you look at the fine print. Which has fine print. Which has...
Which doesn't mean there is not a stable level at which some formal notion of "accurate" cannot be said to exist, which is the minimum viable level of simplification.
Etc.
This sort of thing will of course will the input to many other things.
The current computational and systems biochemistry approaches struggle to model large biomolecules and their interactions due to the large degrees of freedom of the models.
I think it is reasonable to rely on statistical methods to lead researchers down paths that have a high likelihood of being correct versus brute forcing the chemical kinetics.
After all chemistry is inherently stochastic…
For example, I have no idea how a computer works in every minute detail (ie, exactly the physics and chemistry of every process that happens in real time), but I have enough of an understanding of what to do with it, that I can use it as an incredibly useful tool for many things
Definitely interesting times!
Except for weird cases, computers (or cars, or cameras, or lots of other man made devices) are clearly known and you (or another specialist) can clearly show why a device does X when you input Y on it.
And despite having these "best method", it didn't prevent progress in theoretical physics, theory and experimentation complement each other.
ML models are just another kind of model that can help both engineering and fundamental research. Their working is close to the old guy in the shop who knows intuitively what is good design, because he has seen it all. That old guys in shops are sometimes better than modeling using physics equations help scientific progress, as scientists can work together with the old guy, combining the strength of intuition and experience with that of scientific reasoning.
Deterministic methods can predict result with a single run, ML methods will need ensemble of results to show the same confidence. It is possible at the end of day that the difference in cost might not he that high over time.
"produce a process that arrives at this result" should be just another answer it can spit out. We don't necessarily care if the answer it produces is actually the same as what originally happened inside itself. All we need is that the answer checks out when we try it.
As a scientific theory for fundamentally explaining the nature of the universe, maybe it won't be as useful.
In other words I bring a hypothesis to AF3 and ask for it to refute or affirm.
Replace "human-interpretable theories" with "every man interpretable theories", and you'll have a pretty good idea of how > 90% of the world feels about modern science. It is indistinguishable from magic, by the common measure.
Obtuse example: My parents were alive when the first nuclear weapon was detonated. They didn't know that they didn't know this weapon was being built, let alone that it might have ignited the atmosphere.
With sophisticated enough ML, that 90% will become 99.9% - save the few who have access to (and can trust) ML tools that can decipher the "logic" from the original ML tools.
Yes, interesting times ahead... indeed.
I'm sure that similar arguments for and against the proof apply here as well.
We had semiconductors and superconductors before we understood how they worked -- on both cases arguably we still don't completely understand the phenomena. Things like the dynamo and the electric motor were invented by practice and later explained by scientists, not derived from first principles. Steam engines and pumps were invented before we had the physics to describe how they worked.
For this reason, discoveries made by AI will be immensely useful for accelerating scientific progress, even if those discoveries are opaque at first.
What happens when the best methods for computational fluid dynamics, molecular dynamics, nuclear physics are all uninterpretable ML models? Does this decouple progress from our current understanding of the scientific process - moving to better and better models of the world without human-interpretable theories and mathematical models / explanations? Is that even iteratively sustainable in the way that scientific progress has proven to be?
Interesting times ahead.