This is essentially the goal of probabilistic programming[1]: let the programmer specify and tweak the model, and have the analog of a compiler handle inference. Finding a good model is then analogous to debugging.

You are mostly stuck with Bayesian models, since that's what we have general-purpose inference algorithms for. And in practice an understanding of Bayesian statistics is a prerequisite for writing useful probabilistic programs.

[1] e.g. http://probabilistic-programming.org/