News for hackers. Stuff that matters.
Re:Do the English Use More Words Than the French? by pash on Friday February 20, 2015 @05:26PM (#9083489)
Word-use is roughly Zipf-distributed [0] in most all modern languages, so there's not much difference in the number of commonly used words (as long as you ignore basic differences like the existence of articles and the richness of word inflection).

An analysis of English and French texts on Project Gutenburg shows that 93 words account for half of English usage, versus 89 for French [1]. English has fewer words in the middle of the distribution (696 versus 795 at 70%) and in the near tail (6,428 versus 9,050 at 90%, and 14,736 versus 21,231 at 95%), but more in the extreme tail.

0. https://en.m.wikipedia.org/wiki/Zipf%27s_law

1. http://1.1o1.in/en/webtools/semantic-depth

Re:Do the English Use More Words Than the French? by murbard2 on Friday February 20, 2015 @05:59PM (#9083663)
> 14,736 versus 21,231 at 95%

So about 50% more words for French. The very extreme tail is less interesting because it captures the size of the dictionary, and words very rarely used.

I think the exponent of the tail would be the most relevant metric, but I can't open those pdf. Can someone plot the inverse CDF and make a log-log plot?

Re:Do the English Use More Words Than the French? by pcrh on Friday February 20, 2015 @05:37PM (#9083558)
>50th percentile of word use at 93 words for English and 89 for French [1].

If I understand the meaning to that phrase correctly, it seems to be an extraordinarily useless measure