> A bias towards "beauty" or simplicity of some form is inevitable and part of good science.

> For every theory, you can generate a more complicated theory with more moving parts that generates the exact same predictions.

I agree with the second point, but disagree with the first.

Yes, by Occam's razor, a simpler theory with fewer moving parts should be preferred over a complex theory, if both produce correct answers. But the most accurate theory might not be "beautiful".

What matters most is if the theory correctly predicts reality. If it has a weird arbitrary format, but correctly predicts everything, it's better than a theory that's beautiful but incorrect.

Most of science has been using "beauty" as a way to help guide it to the correct answers. For example, there's been a lot of work on string theory because it has a kind of mathematical beauty. But it may turn out that a search for "beauty" in this sense was misguided. Beauty is not the goal, truth is.

"the most accurate theory might not be 'beautiful' "

Welcome to Biology! :-)

(Except DNA -> RNA -> Protein, but it's still sloppy along the way)

> You have to appeal to some principle to sort out such theories from good descriptions of how the world works.

Yes: testable predictions followed by experimental evidence.

Beyond that I don't see how what you're talking about is anything other than an appeal to aesthetics.

> Yes: testable predictions followed by experimental evidence.

But there are an infinite number of theories matching the current evidence. Some predictions of the theories might not be testable, what do you do then?

Simple: you label them as fiction rather than science. The scientific method has experimental feedback built into the loop.

Well then you have to label everything as fiction rather than science because if you don't single out the simplest theory you can't say that general relativity is science and general relativity plus sun turns into cake tomorrow isn't science. And if someone tells you the sun will turn into cake tomorrow and claims that your theory where the sun doesn't turn into a cake tomorrow is fiction and unscientific cause it doesn't make predictions to distinguish it from their theory, you don't have any principled philosophical ground to disagree whith them .

And that's not so clearly bad there because they are obiously wrong, but then you get to complicated topics where everyone has hard to resolve disagreements about which theory should be the default from a big pool of theories that seem to agree whith the evidence.

Doing something clearly outside of empiricism then saying “why doesn’t empiricism cover this” is complete nonsense. What is the ontological explanation for your hypothesis that the sun will turn into cake? Misunderstanding the scientific method is not an issue with the scientific method.