Because Google no longer just searches for literal strings of text, ranked by links.
They now infer your meaning and find results that match your intent -- even if the exact search string doesn't exist in the search results.
For more on this, read about Hinton's work on "thought vectors" [1].
So if Google can't determine the meaning of African American language as well as it can standard English, then that content is likely to be less visible in search results.
[1] https://www.theguardian.com/science/2015/may/21/google-a-ste...
> Some aspects of communication are likely to prove more challenging, Hinton predicted. “Irony is going to be hard to get,” he said. “You have to be master of the literal first. But then, Americans don’t get irony either. Computers are going to reach the level of Americans before Brits.”
Bing seems to do the same thing. I really wish there were a grep for the web because both engines get it wrong even when I try to exclude results.
It's basically grep for the web, since it uses the actual query instead of the meaning of it.
As an aside - this is a slippery slope conversation. Some people believe language requires rules. Other, more special people, think language is fluid and anarchist. The latter of those two do not make search engines.
My guess is that it's unlikely to provide sources that are in one of these dialects, as the link density will be low.
A further problem, though, is that search engines try to understand the query and the content searched at a deeper level than mere keyword matches. For example, they may try to pick up synonyms (if I search for “movies in <local area>”, perhaps results for “films” might be helpful too) and associations.
So that when you search for "cheesecake recipes", you don't get a well-sourced, high-traffic article from a respectable website describing in detail how the new business initiative by Cheesecake Factory is a recipe for disaster.
* Disclaimer: I pulled that example out of thin air. There's actually no such article.
Isn't the ideal system to not assume anything about the content it's analyzing, but to base the model on behavioral patterns instead? In the case of Google ranking, either a site has traffic/low bounce/backlinks/social cred or it doesn't--why is Google attempting to 'read' and analyze the content itself? (Other than to serve ads of course)