News for hackers. Stuff that matters.
Re:Visualizing Tolkien by shardling on Thursday October 04, 2012 @10:28AM (#4612418)
The "originality" index bothered me, because as the work of a length grows, you'd generically expect less words to be introduced -- exactly what the results show.

The idea makes sense, but I'm unclear on how to actually measure it in a way that's normalized by page count.

Re:Visualizing Tolkien by andreasvc on Thursday October 04, 2012 @02:37PM (#4613687)
There's the concept of a vocabulary growth curve. It shows the number of words occurring once as a function of the amount of text, e.g., how much new words in the first 1000 words? how much in first 2000 words? etc.
Re:Visualizing Tolkien by shardling on Thursday October 04, 2012 @02:59PM (#4613768)
But is there a good way to boil that down to a single metric? (i.e., can we parametrize these curves with a single value?)
Re:Visualizing Tolkien by andreasvc on Thursday October 04, 2012 @04:06PM (#4614105)
I wouldn't know off the top of my head, but the concept was introduced by Harold Baayen. There are free PDFs online of his with lots of interesting quantitative techniques.
Re:Visualizing Tolkien by Evbn on Thursday October 04, 2012 @11:55AM (#4612900)
Binomial distribution mumble mumble logarithm of page count mumble.