Aren't Bayesian models essentially statistics + linear algebra? It sounds like you're characterizing ML as something beyond Bayes, which isn't really the case. I don't know what you mean by "toy" problems... that might be kind of true for Naive Bayes, but a Bayesian statistical model can be built in a non-naive fashion and provide useful multi-dimensional probabilities. It's really a kind of underrated technique, though I suppose it's because you can't simply `import Bayes from 'bayes'` to build a non-naive model.

Bayesian models are also advantageous in that they are much easier to describe to stakeholders. Neural networks, on the other hand, are highly difficult(if not impossible in many cases) to describe how they function.