High-reproducibility and high-accuracy method for automated topic classification

Re:High-reproducibility and high-accuracy method for automat... by peter_l_downs on Saturday January 31, 2015 @09:26PM (#8978953)

I built this once! If anyone wants to play around with it the website is http://bookshrink.com and the code is on github (https://github.com/peterldowns/bookshrink)

[ Parent | on HN ]

Re:High-reproducibility and high-accuracy method for automat... by bane on Saturday January 31, 2015 @10:15PM (#8979034)

How did it work? Any interesting tweaks you can share?

BTW, I love the highlight-in-the-original-text option.

[ Parent | on HN ]

Re:High-reproducibility and high-accuracy method for automat... by peter_l_downs on Sunday February 01, 2015 @02:42PM (#8981329)

    > How did it work? Any interesting tweaks you can share?

For being such a naïve approach I think it works fantastically. It works better on some types of texts (newspaper articles, research papers) than others (poems, fiction). It was my first real programming project so the code has a certain sentimental meaning for me :)

The analysis code (https://github.com/peterldowns/bookshrink/blob/master/analys...) is very short and the comments include my thoughts on certain tweaks / approaches. Some quirks of the implementation: uses regex for sentence splitting (really!), doesn't perform any stemming, and weights proper nouns heavier than other words.

    > BTW, I love the highlight-in-the-original-text option.

Thanks! The eventual goal was to build a tool that would make essay grading easier for teachers, although I never got around to it.

[ Parent | on HN ]