News for hackers. Stuff that matters.
Re:The Long Tail of the English Language by jboggan on Thursday January 29, 2015 @04:52PM (#8969099)
It seems like their long-tail data is full of misspellings. Try typing in "vacillate" and looking at the other words on the graph in relative frequency, for example.
Re:The Long Tail of the English Language by eginhard on Thursday January 29, 2015 @05:06PM (#8969181)
No corpus is ever "clean". Depending on the type of corpus there might be many misspellings, so obviously they will occur in the graph alongside other low-frequency words.