What about assigning greater prior probability to hypotheses that are simpler in some sense? Is there perhaps something to that too?
I have an idea of doing a PageRank-style weighting of Turing machines, based on how simple/complex simulating one under another is. That would align with my intuition that most basic universal Turing machines are equally valid, but one that has (say) a short instruction that outputs one particular long sequence of symbols is probably not a valid machine to use.
Is it not scientific method to prove correctness, not simpleness?
The geocentric model required the addition of several epicycle adjustments with no observable cause. Re-framing the local system with the sun at the center eliminated most of the epicycles.
(... and the remaining errors in the heliocentric model relative to the observed motion led to the discovery of the moons and, eventually, Newton's gravitational theory. The driving function, overall, is simplification of the model by removing special cases and the need for acausal "I don't know why; this is what we observe" rules).
Beyond that, the grand truth of what's actually happening becomes a bit uselessly murky. The real function of Occam's Razor is to allow us to simplify models for our own benefit (because simpler models are easier to interpret and use), not necessarily because the simple answer is the Actual Truth(TM), which we can't know outside of observation.
There's a thought experiment in college philosophy: how do you know the universe wasn't created from nothing precisely six seconds ago, with all matter and energy in the universe arranged with the precise concentrations and momenta they would need to become the present universe in precisely six seconds?