Maybe. I take a really optimistic view of attention based mechanisms, like I explained and added to my original post. If you read the recent reformer paper, they produce a new way of computation to start building a model that can in some way, encode the relationships between different parts of war and piece. The bottle neck right now is computation, we don’t know how well these models can learn when that bottle neck is removed!

I’m optimistic because I believe that the contextual information that you are describing, is already there in the vast expanse of the internet.

But I will also add, I think none of this will spawn AI, just that it will spawn new technologies that are categorically different.