Re:Do 20 pages of a book gives you 90% of its words?
by pealco on Friday June 30, 2017 @05:43PM
(#14673768)
This doesn't really address your teacher's claim about having to look words up, though. What you want to look at is the distribution of low frequency words across the book. What do the plots look like when you remove proper nouns, functional words (e.g., "the", "and", prepositions) and, say, the top 1000 most frequent words in English?
[ Parent
| on HN ]
Re:Do 20 pages of a book gives you 90% of its words?
by anon1094 on Friday June 30, 2017 @06:15PM
(#14673878)
Would be very interesting to see this applied to blogs in different categories to rapidly learn languages through reading based on the words that you currently know and the most frequent words in that language. So it would always present you with the article that suits your level and you would have the benefit of learning the most new words.
Re:Do 20 pages of a book gives you 90% of its words?
by _asummers on Friday June 30, 2017 @07:58PM
(#14674355)
Also would be interesting to see it applied to newspapers, with obvious slices like particular author, section (sports v world news etc) distribution year to year, and which paper. TV news broadcasting could also be interesting to compare by the same dimensions, though the conversational style in some interview shows would possibly make this less telling. .
Re:Do 20 pages of a book gives you 90% of its words?
by kiechu on Friday June 30, 2017 @06:17PM
(#14673896)
That's something worth trying.
Re:Do 20 pages of a book gives you 90% of its words?
by mannykannot on Friday June 30, 2017 @08:44PM
(#14674559)
I imagine it would look very much like the plots of unique words given in the article. As you suspect, the chances of coming across one of these is much more evenly distributed.