Re:Responding to jacquesm's challenge
by dangoldin on Friday March 19, 2010 @10:18AM
(#1204168)
Isn't the Naive Bayes classifier biased to users with a large volume of text? Ie if there are two users and one writes 99% of the content it's very likely that that user will be picked as the author for almost anything? At the same time, this may be desired since someone who does contribute a lot on HN may have also desired to have some fun.
[ Parent
| on HN ]
The key calculation is given a word w what's the frequency with which this user uses word w. So that's number of times w occurs / numbers of words that user has used. So it doesn't matter as long as a user has 'enough' text so that they've covered a good portion of the overall dictionary of words in use.
The prior probability is based on the number of comments a user makes. In this case that prior is insignificant because the sample text is large.