You have people who focus on grammar and spelling. But word embeddings collect their insights by taking any sequence of 5 words, taking out the middle word, jumble up the result (technically they express it in a way that ignores order. They're expressed as 1 bit per word, 1 means the word is in the sentence, 0 means it's not. The sequence of the words in the input to the network is completely independent of their sequence in the sentence). And they understand that "king is to man as queen is to woman" and lots of other things.

When going deeper you quickly start to realize a few things : in 90% of sentences the sequence of words does not matter. No, not even if "not" appears before or after the verb (and thus refers to the subject of object of the sentence). Word sequence. Doesn't matter. Which noun you place an adjective next to. It is semantically important. Really important. Every English (or any language I imagine) teacher will hammer the point home again and again. And yet ... it almost never matters, in the sense that getting it wrong will not cause something dumber than a human to misinterpret the resulting sentence. So why do we care ? Social reasons (ie. to fuck other humans, or more generally, to get them to do stuff for us)

It's a weird thing that keeps coming back in machine learning. Humans think their reasoning high level. Yet algorithms that keep track of maybe 2 or 3 variables per individual can predict the actions of crowds with uncanny accuracy. Tens to hundreds of thousands of people, each believing they're individuals and think about what they're doing, take not just the same decision, but with an enormous probability will come to that decision within minutes of each other.

I am reminded of a quote by Churchill. Humans appear smart individually, but it's a trick, an impression, it's a facade, almost an illusion. If they act in group, said intelligence is utterly gone, and they almost always act in dumb ways in large groups, even when they are acting alone. Intelligence is 95% a parlor trick used in conversation, to make friends, or to mate, like a peacock's feathers, and only 5% or less something we actually use to act. So it's purpose, from a species' perspective, is not at all to act intelligent, merely to appear intelligent to others. Second is that everyone, even if they are smart and correctly reason about the world around them, will still act stupid. Without someone to impress, you could have a triple nobel prize, you won't act it. So intelligence doesn't work in an individual, and it doesn't work in most groups. It only works in groups where the interaction of the group has people impressing each other with what they did, with some sort of reward being given for that.