News for hackers. Stuff that matters.
Re:On Chomsky and the Two Cultures of Statistical Learning by macmac on Friday May 27, 2011 @05:03PM (#2593062)
Are Norvig's comments on the "I before E except before C." really vaild? Why would one use a corpus for analysis of the rule, and not a dictionary? It appears to me that "CIE" (P(CIE) = 0.0014) is more common than "CEI" (P(CEI) = 0.0005) because the words that contain the "exception" "CIE" are used more frequently in the corpus than the words that follow the rule "CEI". Once you know the limited number of exceptions (in the dictionary sense) the rule appears to preserve its relevance.
Re:On Chomsky and the Two Cultures of Statistical Learning by jimbokun on Friday May 27, 2011 @05:10PM (#2593082)
I suppose a the most useful corpus for this rule would be spelling tests.