News for hackers. Stuff that matters.
Re:An Efficient Way to Extract the Main Topics from a Sentence by alok-g on Thursday May 09, 2013 @12:20PM (#5681230)
Thanks!

Here is a study on average sentence length showing it at 24 words per sentence: http://ds.nahoo.net/Academic/Maths/Sentence.html

In my own study, I performed analysis on a corpus of a few hundred million sentences of written text and found it at 31 words per sentence.

It would be great if you can point to the use of GPU for parsing.

I had never heard of the bitpar parser, will look into it.

Re:An Efficient Way to Extract the Main Topics from a Sentence by andreasvc on Thursday May 09, 2013 @06:08PM (#5683307)
My average sentence length of 15 came from a bunch of literary novels. The non-literary fiction had even less, at 11 words. There is obviously quite some variance here, but actually it shouldn't matter for a parser because humans appear to parse in linear time (given a well written sentence).
Re:An Efficient Way to Extract the Main Topics from a Sentence by andreasvc on Thursday May 09, 2013 @12:47PM (#5681433)