There is an issue scientifically. I think this point was expressed by Feynman: the goal of scientific theories isn’t just to make better predictions, it’s to inform us about how and why the world works. Many ancient civilizations could accurately predict the position of celestial bodies with calendars derived from observations of their period, but it wasn’t until Copernicus proposed the heliocentric model and Galileo provided supporting observations that we understood the why and how, and that really matters for future progress and understanding.

I agree the how/why is the main driving goal. That's kinda why I feel like this is not depressiong news — there's a new frontier to discover and attempt to explain. Scientists love that stuff (:

Knowing how to predict the motion of planets but without having an underlying explanation encourages scientists to develop their theories. Now, once more, we know how to predict something (protein folding) but without an underlying explanation. Hurray, something to investigate!

(Aside: I realize that there are also more human factors at play, and upsetting the status quo will always cause some grief. I just wanted to provide a counterpoint that there is some exciting progress represented here, too).

I was mainly responding to the claim that these black boxes produce a hypothesis that is useful as a basis for scientific theories. I don’t think it does, because it offers no explanation as to the how and why, which is as we agree the primary goal. It doesn’t provide a hypothesis per se, just a prediction, which is useful technologically and should indicate that there is more to be discovered (see my response to the sibling reply) scientifically but offers no motivating explanation.

But we do know why, it's just not simple. The atoms interact with one another because of a variety of fundamental forces, but since there can be hundreds of thousands of atoms in a single protein, it's plainly beyond human comprehension to explain why it folds the way it does, one fundamental force interaction at a time.

Fair. I guess the interesting thing for protein folding research then is that there appears to be a way to approximate/simplify the calculations required to predict folding patterns that doesn’t require the precision of existing folding models and software. In essence, AlphaFold is an existence proof that there should be a way to model protein folding more efficiently.

People will be depressed because they spent decades getting into professorship positions and publishing papers with ostensible comprehensible interpretations of the generative processes that produced their observations, only to be "beat" in the game by a system that processed a lot of observations and can make predicts in a way that no individual human could comprehend. And those professors will have a harder time publishing, and therefore getting promoted, in the future.

Whether ML models produce hypotheses is something of an epistemiological argument that I think muddies the waters without bringing any light. I would only use the term "ML models generate predictions". In a sense, the model itself is the hypothesis, not any individual prediction.