Presumably, an ensemble of nuclear theorists, particularly those dedicating their lives to making big LQCD calculations, would avoid inverting large ill-conditioned matrices, if that was a clever thing to do.

I know a few QFT types - they tend to be decently intelligent and mathematically capable. Ok, they're more like wickedly smart...

It depends on the computational step. Often we "just" need to compute the determinant, in which case we use a trick. In some other cases, we do only need linear solves. Exact inversion of the whole thing is often prohibitive.