Historically, haven't the more "elegant" models been shown to be correct? The various explanations for planetary motion come to mind as an example.

No.. In fact by the standards of the day, an circular orbit is more elegant. Think about circles within circles and the entire universe revolving around you.

To some extent, models that made less assumptions tend to be more correct.. but I think those are few and far between. We are now trying to explain very small discrepancies (by human scale) and the models are necessarily more complex.

I was thinking of the epicycle in fact. Kind of a hack, I think.

Elegance is matter of perspective. Epicycles are a very elegant way to explain planetary orbits from a geocentric view point, but they are very wrong.

I would agree epicycles are elegant — if they worked. :-)

They work. It's just an approximation series.

There goes the elegance though.

Epicycles can’t be wrong per se, they can approximate any continuous movement. That is what makes it "curve fitting" and "inelegant".

Correct in a narrow range. Newton's elegance failed to accurately predict orbits of even our rather typical stellar system. Good for a start, but hardly something to strive for once we know better. So where do we go next? There is a real chance of artificially keeping ourselves in a local minimum if we don't seriously consider inelegant theories. It'd be neat if someone came up with an elegant theory that accurately described nature from quantum fields to stars. It'd also be neat if someone did that with an inelegant theory.

But that could just be a consequence of biases in the scientific process, right? After all phenomena that can be explained by simple/elegant theories are found, the remaining unexplained phenomena can only be explained by complicated/inelegant theories.