Re:Amount of profanity in git commit messages per programmin...
by scott_s on Tuesday February 22, 2011 @03:30PM
(#2251414)
That is not the case. He wanted to compare curse words across languages independent of language popularity. If he did not collect the same amount of data per language, then he would have two variables: number of curse words and number of commits. Then there would be the danger that a more popular language would have far more curse words simply because it has far more commits.
Re:Amount of profanity in git commit messages per programmin...
by jamn on Tuesday February 22, 2011 @03:26PM
(#2251387)
I wouldn't call it to 'tweak' the data collection. He is simply normalizing the results to ignore the differences in language distribution.
This is normal and has nothing to do with how you choose to represent it.
It would have been meaningless to show any graph or table saying 'Python has the most messages with profanity" if the amount of Python projects is 80% of all the projects out there.
Re:Amount of profanity in git commit messages per programmin...
by waqf on Tuesday February 22, 2011 @05:23PM
(#2251846)
He is right to normalize the results, but parent's point is that he is wrong to do that by modifying his data collection.
He should just collect as many commit messages as possible, then divide the profanity count for each language by the commit message count. Because that has lower standard error [and no more bias] than what he did.
Re:Amount of profanity in git commit messages per programmin...
by masklinn on Tuesday February 22, 2011 @03:30PM
(#2251410)
That's not the case, that's just his personal choice. He could just as well have gone with %age of swear words per commit which would have made the number of commits per language irrelevant (as long as that number was kept above statistical nonsense) and would have yield the same result.
I like how he had to tweak the data collection process to make the visualization method fit.