Think on it in a different way, most of the very revolutionary theories, those that changed how we see the world, were relatively more simple, they were generally simple enough that one fringe could develop them to a point where they shined so brightly as to be next to irrefutable. Things like how the earth might be round, and circle the sun.
We can't expect the same to be true for more advanced fields, we can't say "yeah, this "new/underdeveloped" idea does seem reasonable, but it does not solve everything as well as our existing theory that we've been iterating on for decades, so let's not waste time on that".
Intresting trivia: heliocentrism didn't show "shining brightly as to be next to irrefutable" - it was a fringe idea that could not be confirmed through observation at the time, required some pretty wild (for the time) assumptions - such as stars being very, very far away, to explain why there's no visible parallax from Earth's movements - and went against existing understanding of physics in general (such as, Earth is very big and heavy and bulky, so it's not obvious how could it be moving in circles very fast). Also, IIRC, the predictions made by heliocentric model were less accurate than geocentric ones.
It took astronomical observations with early telescopes to provide data points favoring a mixed geo/heliocentric model, and then further observations, work of Kepler and Newton's theory of gravity for heliocentric model to finally start making sense.
This does serve as an example backing TFA's thesis: some accepted theories, like (then) geocentric model, may be just local maxima - theoretical dead ends. A potential better theory will initially look bad in comparison, it needs work to develop past the accepted one.
> work of Kepler
Yes. One problem of the early Copernican heliocentric model was that it stated that the orbits of planets around the sun were perfect circles. It wasn't until Kepler showed that a) the orbits were actually elliptical and b) the planets speeded up when they approached the sun and slowed down as they moved away that the actual movements of the planets could be more accurately predicted. Until this time, the older earth-centric models with all of the epicycles were 'better', even though totally unrelated to reality.
Obviously it's different for Mercury and Venus, whose orbits are inside ours. They instead switch between being visible in the morning or the evening.
All very complicated for those ancient astronomers!
Yes, but hundred years ago we were "stuck" with another worldview that was explaining everything fine, and i assume ther was a big resistance from estabilishment to addopt new ideas. But then old scientist died, and resistance got weaker. So we might look back at today after 100 yaers and see similar situation. No one says that ideas of 100 yaers ago were all wrong, just not so true as current.
While worldviews can be overturned and paradigms can be shifted, it is (significantly) harder than it was before. We simply know more now, and have a better understanding of our limits. So whatever new framework that has to supercede our current framework has more ground to cover than it did even in the recent past. This is exacerbated by the fact that in physics in that most everything outside the early universe and black holes (which aren't easily accessible experimentally) seems to conform to our current framework -- there is both more to fit in and less data to work with.
Likewise quantum mechanics doesn’t replace General Relativity. It compliments it.
It doesn't replace, rather complements it.
Occam's razor AND semantics.
Edit: maybe you were joking? in which case sorry (it missed the mark)
Hum... The old scientists that (very vocally on that case) resisted change were the same that uncovered the problems with the old models and laid out the first theories on how to fix those problems. Those things are way more complex than simple quotes and labels can communicate.
New models got adopted when after a lot of work people created some that worked better. Not a moment before. Those better models didn't get resistance from the established physicists.
There's also been the - likely incorrect - belief that different models are too hard to distinguish experimentally.
So there's been a process of continuous refinement of existing theories which are known to be incomplete, and no concerted and sustained attempt to solve foundational philosophical problems - which is the level that Einstein, Newton, and other pioneers operated at.
An alternative explanation for quantum foundations being in limbo is that it is extremely difficult to come up with alternatives that offer a possibility of verification.
Update: writing this reminded me of [1], in which a simple experiment by Shahriar Afshar, that arguably challenged one tenet of the Copenhagen interpretation, provoked a disturbingly over-the-top response, which supports your position on how work on quantum fundamentals is opposed (though, personally, I doubt it succeeds in challenging the Copenhagen interpretation. Interestingly, the opponents of Afshar's interpretation do not all agree on why they think it is wrong.)
[1] https://www.newscientist.com/article/mg19325915-400-quantum-...
Part of the authors premise is that a more correct theory could have less predictive power out thr gate and might not be pursued as a result.
If hard sticking to Ockham’s razor was true, Quantum loop Gravity, String Theory and many other theories wouldn’t be intensively studied for past 50 years. Or development and studying interpretations of quantum mechanics (which btw yielded results in Quantum Information Theory).
It’s just that constructing something correct and new IS really hard.
The so-called "wave function collapse" isn't really part of QM, it's duct tape that we have applied to stick together the QM with "Classical Physics" and our pre-existing assumptions about human consciousness. I don't think we should consider it "real" or "true".
Without the wave function collapse, there is no randomness in QM.
You can't just wave away the collapse mechanism, what do you make of the double alit experiment? isn't the target "real" enough?
That's why the wave function "collapses": Because it collides with the non-quantum target.
It's a useful approximation, but of course in reality, there is no such thing as a non-quantum thing.
And if you evolve the target's wave function with the wave function of the particle, then there is no stochastic collapse.
But would that not also imply that we should be able to measure the quantum world with quantum devices? Say we have a quantum property that is extremely close to p=0.5. If we could invent a device to replicate that property perfectly and measure it repetitively we could then estimate ever more accurate boundaries for the "true" value of p, no?
It's either inherent randomness or just a deep hole in the whole thing (similar to the alien chess thought experiment problem). Personally I choose to believe that the theory is just incomplete because nobody can even define what a "measurement" really is, meaning in which cases what we do is a "measurement" and in which cases it is not a "measurement". I also think that this is what people like Feynman refer to when they say things like "nobody understands QM", it's actually "nobody understands the wave function collapse", the rest is just maths.
The measurement is the theoretical duct tape between the "quantum world" and the "classical world".
But there is no such thing as a "classical world", it's just a useful approximation.
And therefore, there is also no such thing as a "measurement", it's also an approximation.
(Maybe not even an approximation, but maybe more like a projection...)
My layman feeling wrt. QM, and Copenhagen school in particular, is that we're searching for too computationally simple mental models. Most other areas of physics - like GR, SR, thermodynamics - can get away with aggregating matter into points, perfect spheres, etc. because they're working in macro scale, but QM is trying to deal with the smallest bits of our reality. Now the boundary between QM and "classical physics" is one where your quantum system will interact with 10^{double digit} amount of other quantum-relevant bits. I have a feeling that searching for what constitutes "a measurement" in such scenario is missing the point, and even talking about the macro system being entangled with the test system is pretty much skipping over all the interesting bits.
Isn't that the observer becoming entangled with the measured system?
The definitions I have found always invoke the presence of a "classical system"/"observer".
But that just kicks the can down the road, because there is no well-defined definition of a "classical system" either.
(It would help tremendously if we ever measured quantum states that weren't "collapsed", but as we've never done this so far it makes most of the stochastic collapse stuff hard to justify, even if it seems intuitively like the right approach).
Bell's inequalities show that there isn't a local state that can be there.
On the other hand nobody has any clue what the quantum measurement / wave function collapse actually is. There are theories/interpretations but no truly satisfying answers in the same way as for example Newton's equations were a satisfying answer to the elliptical movement of planets, even though we later found out in the 20th century that F ~ 1/r^2 was actually an approximation.
We simply don't know, and we have no idea when shall we know.
Edit: To give one example of an approach that I think is promising: We start by describing the observer and environment through a density matrix (a probability distribution over possible wave functions) and introduce an interaction with a quantum system (e.g. a spin). Given a reasonable interaction, you can show that the entanglement in the combined state (observer, environment and spin) leads to the system approaching a state that is a probability distribution of entangled states where each probability corresponds to the Born rule. Interestingly in this case the probabilities emerge from our lack of knowledge about the microstate of the observer/environment, so it’s actually thermodynamic uncertainty.
I am not particularly a many-world proponent, but I do not think it is fair to level this accusation as an issue for many worlds without bringing up that every other interpretation has the same "flaw".
Although its really cool I don't think Gleason helps you tie any particular interpretation to the Born rule, since you still have to make a jump to tie your measurement outcome to a POM/POVM element.
As far as your last sentence goes, this is sort of what I was trying to argue in my comment above. The "part that is difficult" that you identify as being unresolved by MWI is also completely unresolved by pilot wave theory, or qbism or consistent histories or any other interpretation (as far as I am aware).
QM is a model of "what we can observe from the world" based on "what we can observe from the world".
QM is a model of "accessible" information.
What laymen usually wants is to understand how the world evolve.
What QM physicists tell them is there exist some inaccessible information, but using accessible information we have, we know how to predict all the accessible information (albeit stochastic-ally).
The typical example to help computer scientists to understand is the seed of a pseudo-random generator in an online casino. The players will never be able to access the seed, therefore the best they can do is make decision based on the value of the generated random numbers they observe and their probabilities.
Bell inequalities are a consequence of this modelisation. They are a refurbishing of Boole's inequalities, a theorem about probability which only bound those who use probability.
The usual fallacy forward is telling QM is a non-local theory, classical local theory can't violate Bell inequalities, Bell inequalities violations are observed in the real world, therefore the world is not local...
If you can prove conclusively there's no randomness and no collapse and no need for either, a Nobel Prize awaits. If you can't - most likely - one opinion is as (in)valid as any other for now.
Of course while it’s good to search for alternative models/theories/explanations, unless you can provide something with better with more predictive power than the existing/widely accepted ones, it’s a good idea to hold the critique.
EDIT: To clarify: by less predictive power I mean that it neither explains new effects or predicts new unknown ones, nor explains known phenomena or generates existing theories as special examples. I didn’t mean theories such as for example string theory, that has little predictive power at the moment, but has current theories as special cases and holds the promise of explaining things that current theories cannot. /EDIT
Physicists are “stuck” with existing theories not because they like them, but because they work so well it’s hard to invent something that even works equally well (not to mention something that works better). There a lot of smart that are brave in thinking and propose wild explanations. Yet, in most cases they don’t stand up the test of time.
Einstein couldn’t deal with randomness of Quantum Mechanics, put forward a hidden variable theory and it was (and still is) seriously considered, but he (and many others) weren’t able to put forward better-working theory. We stick to QM despite its weirdness/randomness because it works extremely well, not because we like it or think things must be this way and require no further study/“it is the most efficient and parsimonious possible model“.