Ask HN: Libraries for classification of text
I've searched fairly hard, but I'm finding it hard to locate exactly what I want. Maybe it doesn't exist, and maybe I just need to adapt something that's there, or write my own. But I thought I'd ask.
I have a bunch of text items that have had classifications applied to them. These classification aren't exclusive - each item may be in one or more classes. Perhaps it's best to think of them as tags.
What I want is a system that has a stab at applying these tags to previously unseen items, based on the tags of existing items. I know that such things exist, but they all seem to be poorly performing, proprietary, hard-to-use, free-but-closed, or otherwise just not quite what I want.
Can I take suggestions from the floor? Is this a problem you've dealt with, and what did you use?
TIA.
(
5 comments
| 3 points
| on HN
)
Or if you want something simple I wrote about how to create a naive Bayesian text classifier in Dr. Dobbs some time ago: http://www.drdobbs.com/tools/184406064
I have written a Bayesian classifier before, but that was binary, not multi-label. I can run multiple binary Bayesian classifiers, but that's probably not the best thing to be doing.
Hence the question.
I'll check your suggestions - thanks.
Added in edit: The package "bow" doesn't compile on my machine, so now I'm looking at an undetermined amount of time fixing the package so it does compile. Always fun.
Looks like I'll have to re-invent the wheel.
As far as my experience with NLTK and machine learning in general goes: you'll need to spend a fair bit of time figuring out which features to extract.