I think there's value in having a huge learning set, but there are also limitations, as you're alluding to.

But I also don't think we're miles away from computers learning some sort of reasoning structure. Some sort of causality-type thinking where you have hierarchies which at some level are reasonably simple, because humans can only fit so much. But when the computers figure it out, they don't have that problem.

At the moment, yes, there's a huge corpus of patterns and you can make some smart decisions just by being able to learn from the huge library, but it's the difference between knowing that one move tends to beat another, without knowing the why. For instance in sports, you have man-marking vs zone-marking. The naive thing to do is just tabulate how often a team did one or the other vs how often they won. Then break it down even more by who they were facing and various stats. But if you don't have a theory of marking, you're a bit lost for explanation, even if the tabulation clearly says zone marking tends to win. A causal explanation might sound something like "man marking allows the other team to pull you out of shape and gives them the choice of which players face which". It might also tell you that sometimes it's actually smarter to man-mark, eg when there's some player you really feel is dominant and needs to be taken out.

I gather that people are working on this causality type AI though, so no doubt we'll see something interesting soon.