News for hackers. Stuff that matters.
Re:A.I. Doesn't Get Black Twitter by a3n on Saturday September 24, 2016 @02:49PM (#12572044)
There may be some of that, but I think it has more to do with AI still being young, and it's easier to work with less ambiguous text for learning. That and everyday speech move much faster than formal writing. Now that my son has gone off to college and is less accessible as a resource, I expect to almost totally lose touch with the speech of "these kids today."
Re:A.I. Doesn't Get Black Twitter by ralfd on Saturday September 24, 2016 @03:18PM (#12572165)
> I expect to almost totally lose touch with the speech of "these kids today."

Is this long term really a problem though? Regression to the mean: No one cares about the specific slang of the 60s anymore. And the hip slang of today will be out of fashion in 15 years.

In a similar vain the extreme T9 keyboard text message abbreviations of the youth in the 2000s did go out of fashion because of automatic spell checking and speech recognition of modern smartphones.

Re:A.I. Doesn't Get Black Twitter by thaumasiotes on Saturday September 24, 2016 @07:32PM (#12573118)
> Is this long term really a problem though?

How long term? What's "a problem"? We know with certainty that given the passage of enough time, the speech of "these kids today" will be completely unintelligible to you.

Re:A.I. Doesn't Get Black Twitter by gugagore on Saturday September 24, 2016 @04:36PM (#12572541)
Do you think written Standard English is less ambiguous than any other English, or written language? Because I don't see why that would be the case.
Re:A.I. Doesn't Get Black Twitter by Dylan16807 on Saturday September 24, 2016 @07:33PM (#12573121)
>Do you think written Standard English is less ambiguous than any other English

Definitely. Things are clearer when you write formally and strictly adhere to grammatical rules. When I'm speaking casually I omit words and (ab)use punctuation much differently, and while that is perfectly acceptable to do, it is more complex and ambiguous to parse.

As far as I'm aware the same pattern exists in a lot of languages and dialects, and it's possible to define formal AAVE too.

Re:A.I. Doesn't Get Black Twitter by Dylan16807 on Sunday September 25, 2016 @01:44AM (#12574143)
To your comment that was detected as a duplicate:

>I don't think there is anything "formal" about Standard English.

"Standard" English as she is spoke? No. But a3n was specifically talking about the formal version often called "Standard written English". It is very prescriptive and less flexible, which makes it easier to parse.

> Can you give an example of what you're talking about? Which grammatical rules do you have in mind?

Let's start with "every rule involving punctuation".

Re:A.I. Doesn't Get Black Twitter by gugagore on Sunday September 25, 2016 @03:56AM (#12574386)
I'm not trying to be dense, but I still don't see it. Punctuation helps with identifying boundaries of clauses, whether they're dependent or not, whether a sentence is a question. Is this what NLP struggles with? I don't think so. One of the big unsolved problems is understanding the referents of pronouns (and other proforms, like "I will [do so] too"). Style guides and generally prescriptivism does not help that.

Another ambiguity is parse ambiguities like " I saw the girl with the telescope." What does "formal" English say about that?

Re:A.I. Doesn't Get Black Twitter by a3n on Sunday September 25, 2016 @04:49PM (#12577360)
That's where AI comes in. Unless there's a human at the ready.
Re:A.I. Doesn't Get Black Twitter by morgante on Saturday September 24, 2016 @07:56PM (#12573170)
> Do you think written Standard English is less ambiguous than any other English, or written language?

Absolutely. It follows a strict set of formal rules and you can readily find sources to learn those rules. Heck, copyeditors regularly debate the specific nuances of how to consistently write Standard English.

Other dialects (and spoken language) are far less formalized. Where is the AP Stylebook or Strunk & White for AAVE?

Re:A.I. Doesn't Get Black Twitter by gugagore on Sunday September 25, 2016 @01:33AM (#12574114)
Can you give an example where Strunk and White or a style guide vhelp to disambiguate some expression? I can't think of any.
Re:A.I. Doesn't Get Black Twitter by morgante on Sunday September 25, 2016 @01:45AM (#12574146)
The Oxford comma is the most obvious case I can think of.