Hacker Hacker News - see just the programming/math/science links from HN

Re:Hacker Hacker News - see just the programming/math/scienc... by patio11 on Monday July 06, 2009 @02:23AM (#689233)

I worked in Bayesian spam filters for a while at the last day job. Wonderful things, aren't they?

I've got to say, your classifier looks like it is amazingly accurate for those distinctions. I'd be really interested in seeing some of the very determinative words. (There are some obvious candidates like "Erlang" and "Twitter" but every time I do a natural language processing project I'm amazed by how our distribution of "little meaningless words" changes markedly between FeatureA and FeatureB.)

[ Parent | on HN ]

Re:Hacker Hacker News - see just the programming/math/scienc... by sqs on Monday July 06, 2009 @02:39AM (#689248)

Yeah, it's amazingly accurate for now only because the training data consists of a high percentage of the actual data set, and so it's probably classifying based on those insignificant differences. We'll see how it holds up...

[ Parent | on HN ]

Re:Hacker Hacker News - see just the programming/math/scienc... by 10ren on Monday July 06, 2009 @04:42AM (#689347)

I think you're training it on the submission titles - I wonder if the text of the websites themselves might be more accurate. Certainly richer. But it's quite possible that the submission titles are more accurate.

[ Parent | on HN ]

Re:Hacker Hacker News - see just the programming/math/scienc... by jlees on Monday July 06, 2009 @05:49AM (#689402)

Says above he's using body text.

[ Parent | on HN ]