FYI, some folks (see: https://spectrallearning.github.io/icml2014/) are certainly working hard to apply eigenvalue decompositions (well, more like generalized SVDs) to any inference problems they can get their convexity-loving hands on.
Disclaimer: my lab mate has been organizing these workshops for the past couple of years and I'm currently procrastinating from preparing a presentation on this material for a reading group later this week, so I've presently got mixed feelings towards it.
Hmm, not in general. (I've been working on various problem-specific versions of the backwards case for most of my career.)