But they do produce knowledge, just not in a form humans can easily reason about. We may not know how AlphaGo plays Go, but I think we can say AlphaGo knows how to play Go. The equivalent applies to AlphaFold and other algorithms that are producing useful knowledge.

Edit: Uh oh. HN may not be ready to accept this, so maybe it’s less progressive than I thought. The metaphysics of the Enlightenment went out the window a long time ago. Functionalism does not require Reason. Knowledge isn’t necessarily anthropocentric.

Not saying I like this either, but it’s certainly the transhumanist future we’re currently charting. Personally, I think we’re peering into a nihilistic abyss and need to fuse a metaphysic back into the loop to get back on the right track.

It's more of a philosophical question really, but are AlphaGo and AlphaFold "producing" knowledge? AlphaGo plays go, AlphaFold predicts protein structure. They are useful tools, but I wouldn't call what they produce knowledge. However, as you say, their underlying models have certainly captured knowledge that we, as humans, do not comprehend yet.

So I would reformulate the observation of the article as follows: we have become extremely good at building useful tools that embeds knowledge that we do not understand.

If I may attempt an analogy, it reminds of Ptolemaic tables: quite useful at predicting planetary motions with some accuracy. What they lacked in understanding, they made up with accumulated data and complexity. Maybe AlphaFold is just that: a Ptolemaic table built on top of a huge swath of data.

We will probably build a lot of these tools in the future, as we have very powerful tools to build them. But I don't think they will replace a more fundamental understanding (that is to say science), that will simply lag behind.

If someone or something produces knowledge, then yes, they are scientists and they need not be human. However it must at least be intelligible to them, and in that case it is that entity doing science.

ML systems as they exist do not produce knowledge for humans, and they do not understand anything themselves. They're mechanisms that do work. A chess computer doesn't understand chess any more than a lever understands physics, even if they can beat humans or lift heavy objects. Your TI-83 is not a mathematician despite the fact that it can calculate much quicker than you can.

If you're suggesting that at some point we may built some sort of AGI-scientist that can actually reason about something, potentially at a level we can't understand, sure. But that is not what ML is, that is science fiction, and it is not how ML systems will be used in the scientific process, they will aid as tools as they're doing right now, and that is what everyone will tell you who worked on AlphaFold.

You could say the same about humans; that we do not understand anything ourselves; that we’re mechanisms that do work. But it’s a moot point.

What I’m saying is that “reason” is an anthropocentric concept and is not a prerequisite for the production of knowledge, or even conducting science for that matter.

Reason isn't an anthropocentric concept at all. Knowledge isn't even produced. Knowledge is the subjective representation of facts, objects, skills and so forth. What's being produced by something like Alphafold is data, interpreting that data by means of reason and abstracting it into a model is what we call knowledge. The entire process is science.

And the first point you make is very important. The fact that we have neural nets in our heads that we didn't and don't really understand hasn't changed anything about what science is. We reason about the world and we do science because having general models about the world we are in is more powerful than merely collecting or correlating data. And that will be the case for any kind of intelligence.

We don't have a perfect model of reality to do science completely virtually using unsupervised ML. Science by definition is not mathematics as it is grounded first and foremost in empiricism. You can use ML to help produce science. E.g. hook up a sufficiently sophisticated "AI" system to do experiments and analyse data and generate hypothesises. But the AI itself won't obsolete the scientific method. I am surprised that the posted article is written by a science journalist, I suspect it is mostly due to her not understanding how ML is implemented rather than misunderstanding the scientific method.

> But the AI itself won't obsolete the scientific method

I wouldn't put a lot of money on that in the future given that even 3 years ago AI could prove about 58% of a set of theorems: https://mathscholar.org/2019/04/google-ai-system-proves-over...

From that to inferring then proving new theorems in the future or even inferring new theories in other fields (that can then be tested empirically if needed) I think we're not very far from what the person you replied to suggested

However, it just modeled a known algorithm for solving theorems mechanically. 3-SAT predates ML, and it was trained using it. Essentially parroting an algorithm is not science.

Call me again when AI develops a new theorem from scratch, in a concise manner

We have a tendency to reify science as something sacred or uniquely human, but the scientific method is essentially an algorithm. Like the article said, AI does not need to develop theorems in a post-theoretical mode of science. AI only needs a systematic way to make discoveries that are in the self interest of the system in which it's embedded.

Humans may need theory to perform science, but only because we're slow and have bad memories is it necessary for us to rely on faint signals like intuition, eureka moments, and pulling all-nighters in the lab. Machines don't need narratives and metaphors to represent their knowledge of the world, nor do they need reason or explanations of how and why. They just need to function.

Or it may be that the author has an understanding which you are lacking.

> We may not know how AlphaGo plays Go, but I think we can say AlphaGo knows how to play Go.

To be honest, before I can agree or disagree with that, I'd really like to know more about how AlphaGo actually plays Go. Like, how the network is structured, how an actual decision is produced, etc.

I feel, we're talking about increasingly lofty and abstract concepts while we don't have much of a clue what actually goes on on the ground.

Oh boy, just wait until you discover that all of science is based on induction

Is it though? In science one uses deductive, inductive and abductive inferences right?

The laws of thermodynamics, for example, just like all the laws of physics, are just consistencies we have observed not breaking after decades of trying every corner case we can conjure up. They are quite literally the definition of inductiveness: you see that something holds for a solid number of cases so you assume that it just holds for everything else too. Of course any scientist who hasn't just memorized a bunch of formulas but has actually grasped the reasoning behind science will tell you that even the most fundamental law of the universe has a possibility of being false, just like the Riemann Hypothesis can't be considered proven just because it works for an unfathomable amount of computed values.

Science is about creating models and then studying them and making predictions off of them. The making of the model is an inductive and empiric ordeal, whereas working on said model is a matter of deduction.

Karl Popper just turned in his grave.

Science does not produce knowledge. Science tries to attach a model for people to reason about what we observe. Its like solving x1 + x2+ ...+ xn= 10, where we dont the extent of what x values can be, or even what operations are being used, all we can observe is the result: 10. Science is saying it could be n=2 and x1 is 3 and x2 is 7.

That isn't knowledge, we as a species can never actually know anything, since that's saying that some things can never be proved wrong, which goes against the philosophy of science.

Fact is the enemy of truth, knowledge the enemy of science.