While I agree with the general premise, don't you think that if we were to find a simple explaination that elegantly explains many things at once, it would be at least worth a try to find ways of falsifying that explaination?

Not at any cost of course, but the falsifications gathered e.g. in trying to disprove the string theory might also help us figure out what is actually going on.

Sure, but once it keeps failing for a few decades, and given that ST is significantly more complex than either QM or GR, not simpler, there has to come a time where it simply is abandoned, even if it hasn't been disproven.

A few decades…

How long did it take to go from Principia to General Relativity?

Principia was proven right at every turn, and explained all of the observed mechanics experiments, all the way up until Maxwell's equations were formalized and their effects tested out, some 200 years after it was published. It's also important the theories it replaced were massively more complex.

Maxwell's equations were first formulated in 1865, the "patch" to Newton's theories was formulated soon after (the luminiferous aether), the Michelson-Morley experiment proving the patch did not work was run in 1887, and the theory of special relativity was proposed in 1905 - so it took about 20 years at most between Newtonian mechanics being conclusively proven to contradict experiments and a new theory becoming proeminent. And special relativity was quickly replaced with general relativity (just 10 years later), because, despite its success, its limitations were immediately apparent.

What's the difference between electromagnetic field and ether?

As far as I understand it, the ether was supposed to be matter, ultimately made out of some kind of particles that obey laws of motion.

Conversely, the electric field is just a potential. It's not made up of anything, it just describes how charged particles interact if they are in a particular place in space-time.

Pretty sure electric field is material. Interaction is described by theory, not by field, field is material nature of interaction.

Yes, but we shouldn’t pull that theory out our mathematical asses. We should always be falsifying things not trying to make an unfalsifiable theory work because the math is interesting.