One of the things that seems to be going on is that, after relatively limited visible progress for a long time, the ability to collect and analyze large datasets suddenly produced fairly striking results. And this in turn has led to a lot of thinking along the lines of "We just need to collect more data and crunch the numbers a bit better."

As you suggest, for at least some classes of problems, we'll find that this approach will asymptotically approach some level of "good" that turns out to not actually be good enough. And the ways to get beyond that will be hard and may not be obvious.