Actually in real science theory is first, observation comes later as a way to judge competing theories.

This is because all observations are theory-laden. They don’t tell you anything without a framework to interpret them in.

Isn't this an idealization myth of the scientific method? I'm pretty sure some kind of observation prior to forming the theory will account for many theories.

True that. It is like cat and mouse. You present a theory and validation and usefulness will come from experimental verification. And some theory comes from unexplainable observations. Michelson–Morley experiment and special theory of relativity will be taking the second path.

In a lot of situations, the observation is that the current favorite theory doesn't work for some observation, but the new theory explaining it only comes much later.

I disagree because we don't entertain theories which make no testable predictions. It seems logical that this rule makes observation the ultimate qualifier.

Yeah, as long as there's no experimental confirmation, it's metaphysics, which is a sub-branch of philosophy, not science. (And it doesn't matter how refined their mathematics are...)