Though I see what you're saying, using a PPL for VAEs just seems like overkill given the simplistic nature of VAEs.

PPLs are useful when the data generation process is not easily represented by something like a simple multivariate Gaussian, etc. You find many good examples academic research, e.g. epidemiology.

Yes but mathematical integration is difficult, the higher the dimension, the more difficult it is. That's why differentiation is preferred. The concepts behind PPL are firmly entrenched in probabilistic ML, the ideas were never lost.