News for hackers. Stuff that matters.
Re:Stanford Natural Language Parser by YeGoblynQueenne on Saturday January 21, 2017 @01:43PM (#13451399)
My favourite example phrase to mess with NL parsers:

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

Stanford fails miserably on this one, even at the tagging step:

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/JJ
  buffalo/JJ
  buffalo/NN
  Buffalo/NNP
  buffalo/NNP
... but then, most humans also find that sentence really hard to parse.

Check it out:

https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...

Re:Stanford Natural Language Parser by izacus on Saturday January 21, 2017 @04:06PM (#13452159)
I doubt most english speakers would correctly parse this sentence either, so I'd say the results are human-like? :)
Re:Stanford Natural Language Parser by WildUtah on Saturday January 21, 2017 @09:23PM (#13453242)
Most English speakers can parse the sentence just fine when they hear it spoken.
Re:Stanford Natural Language Parser by jholman on Saturday January 21, 2017 @11:46PM (#13453624)
Not so. I've tried this on a few dozen students (post-secondary, many of them post-baccalaureate, perhaps 50% ESL) over the years; of those who have not heard it previously, all others say "what? that's not a sentence". I don't recall a single counter-example.
Re:Stanford Natural Language Parser by khedoros1 on Saturday January 21, 2017 @11:49PM (#13453634)
That's surprising to hear. I don't think I've ever found someone that understood it without an explanation. I like using it as an example of how confusing English can be to parse.
Re:Stanford Natural Language Parser by WildUtah on Sunday January 22, 2017 @07:50AM (#13454963)
English is tough, though through thorough thought it can be understood.
Re:Stanford Natural Language Parser by YeGoblynQueenne on Sunday January 22, 2017 @04:38PM (#13457683)
Shouldn't that be "English is tough, though through thorough thought it can be taught"?
Re:Stanford Natural Language Parser by eriknstr on Saturday January 21, 2017 @10:02PM (#13453368)
I would not have been able to tell which buffalos were verbs, which were nouns and which were proper nouns without the rephrasing of the sentence on Wikipedia. In fact even with that rephrasing I still find it difficult to tell which buffalo in the original sentence is what.
Re:Stanford Natural Language Parser by utopcell on Saturday January 21, 2017 @10:00PM (#13453363)
the argument is nevertheless valid: the parser does not process any additional audible signal
Re:Stanford Natural Language Parser by xtiansimon on Saturday January 21, 2017 @02:25PM (#13451607)
That's great. But let me ask, technically, how is the sentence "a grammatically correct sentence in American English" without punctuation? For NLP, shouldn't this first be regarded as a 'special case' of unpunctuated sentences? (and BTW, the parser doesn't handle the punctuated version any better).
Re:Stanford Natural Language Parser by sthatipamala on Saturday January 21, 2017 @03:28PM (#13451935)
It is in fact grammatically correct without punctuation, just somewhat awkward. Consider this analogous sentence with the same meaning, with optional parentheticals added to help you parse:

New York oxen (that) New York oxen bully (themselves) bully New York oxen.

Re:Stanford Natural Language Parser by gcb0 on Sunday January 22, 2017 @03:09AM (#13454217)
the fact that nobody can understand without a preposition and a pronoun even after you replaced the uncommon words with common ones tells me this is a bunch a buffalo excrement that barely entertain linguist phds.
Re:Stanford Natural Language Parser by jcranmer on Saturday January 21, 2017 @05:34PM (#13452477)
That sentence is virtually the same parse tree as "The man I saw crossed the street," which is a totally reasonable sentence to most English speakers. The difference in the parsing is that the "I" in this case corresponds to "Buffalo buffalo", a noun phrase--and the sentence is confusing because we have a tendency to add the "that" for noun phrases instead of single-word noun/pronouns, which the sentence is missing.
Re:Stanford Natural Language Parser by yongjik on Saturday January 21, 2017 @06:25PM (#13452685)
Crucially, what is a reasonable sentence depends heavily on words.

"History lamps traffic people house cry Michigan algebra" also has the same parse tree, but if I utter this, people will probably think I need medical attention.

Re:Stanford Natural Language Parser by YeGoblynQueenne on Saturday January 21, 2017 @06:11PM (#13452629)
For me it was confusing the first time I read it because I didn't know that "buffalo" can mean "to bully". Without that information it doesn't make sense at all.

So I never thought there's something inherently difficulty in the syntax of the sentence- I thought it's just that unexpected third use of "buffalo" that makes it so strange.

Of course, now that I think of it, what you say makes sense also. For an NL parser, the variant use of "buffalo" shouldn't be an issue. You could even say that lacking the context of what a "buffalo" really is, a parser should even find it easier to parse the sentence than a human, who must pause to try and understand what it means.

Maybe it's not such a bad sentence to test parsers after all.

Re:Stanford Natural Language Parser by WildUtah on Saturday January 21, 2017 @09:25PM (#13453248)
'Buffalo' doesn't mean 'bully' in English. It means 'con.'
Re:Stanford Natural Language Parser by eriknstr on Saturday January 21, 2017 @10:04PM (#13453376)
The Wikipedia article above says that though uncommon in usage, the buffalo verbs in that sentence mean "to bully, harass, or intimidate" or "to baffle".
Re:Stanford Natural Language Parser by jholman on Saturday January 21, 2017 @11:48PM (#13453631)
AIUI, it's only "to bully" in the sense of verbally overwhelming someone into believing what you what them to believe. It's like the word "to snow" (in the sense that means "to con").
Re:Stanford Natural Language Parser by douche on Sunday January 22, 2017 @12:09AM (#13453701)
I would read "buffaloing" somebody as a less vulgar version of bullshitting, which is also a way of lying about something, mostly to get them to buy something they don't actually need.
Re:Stanford Natural Language Parser by YeGoblynQueenne on Sunday January 22, 2017 @04:43PM (#13457709)
I don't think "buffalo" (the verb) having a specific meaning makes any difference in how hard that sentence is to parse.
Re:Stanford Natural Language Parser by 0x54MUR41 on Sunday January 22, 2017 @03:29AM (#13454266)
I don't know if that sentence is a grammatical correct sentence in English. Moreover, I don't know what the meaning of the sentence.

I wonder how the other NLP tools parse that sentence. These are the results:

* Spacy [1]

  Buffalo/NNP
  buffalo/NN
  Buffalo/NNP
  buffalo/NN
  buffalo/NN
  buffalo/NN
  Buffalo/NNP
  buffalo/NN
* NLP4J [2]

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP - pos2=CD
  buffalo/NNP - pos2=CD
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP
Well, that sentence really makes NLP tools confused.

[1]: https://spacy.io/

[2]: https://emorynlp.github.io/nlp4j/

Re:Stanford Natural Language Parser by tdmule on Saturday January 21, 2017 @11:17PM (#13453559)
I just found out that Alexa has a funny response to this sentence.
Re:Stanford Natural Language Parser by sean_patel on Sunday January 22, 2017 @12:12AM (#13453715)
Funny, but jokes aside, I found the NLP to be quite impressive / accurate.

I put the following phrase

> I nut on my girlfriends face.

and the parser correctly tagged 'nut' as a VBP (i.e. Verb, non­-3rd Person Singular resent) i.e. that 'nut' in this context is a verb, and not a noun.

Here's what the tagging returned (all words with tags)

I/PRP nut/VBP on/IN my/PRP$ girlfriends/JJ face/NN ./.

Tag Descriptions: http://stackoverflow.com/a/1833718/325521