In virtually no cases that I can think of does modern machine learning approach statistical inference.
Statistical inference is always concerned with estimating confidence in beliefs and typically more concerned with the confidence in the model parameters rather than the model predictions.
Machine learning is almost exclusively concern with model predictions and the performance of these predictions.
You can easily have a model that is very useful for statistical inference but would be awful for machine learning, and most machine learning models (especially neural networks) as useless for statistical inference but great at prediction.
This is not a critique of machine learning or vice versa, its just the case that these two approaches to modeling are quite different and used to solve very different problems.
Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations.
It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.
Can't a RF or boosted regression tree theoretically do the same and then some (approximate discontinuous functions)?
Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem.
The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".
Can anything?
Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
Could you give an example of a halting problem that humans can solve?
> it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
Do you have supporting evidence?
You aren't wrong. Modern Machine learning is entirely statistical inference, mostly through minimizing an objective function.
> Wouldn't this have a limit on functionality?
A neural net can approximate any continuous function (from one euclidean space to another)
https://en.wikipedia.org/wiki/Universal_approximation_theore...