Overview of Text Extraction Algorithms

Re:Overview of Text Extraction Algorithms by buss on Saturday March 19, 2011 @02:12PM (#2344275)

I am working on a project (more of a public service than a startup) that needs this. I've looked through all of the resources linked in the articles above and nothing works as well as I need it to. The best performer is readability, so I will probably be going with the python port of that.

If your code works well I also think you should put it up on github. You can see what I intend to use this technology for by reading this text snippet: https://github.com/sbuss/revisionews/blob/develop/web/index....

[ Parent | on HN ]

Re:Overview of Text Extraction Algorithms by petercooper on Saturday March 19, 2011 @03:29PM (#2344459)

Currently in the middle of a re-architecture/re-write due to its flaws but something similar to this in Ruby I worked on last year: http://github.com/peterc/pismo

[ Parent | on HN ]

Re:Overview of Text Extraction Algorithms by buss on Saturday March 19, 2011 @09:24PM (#2345225)

This looks pretty great, I'll definitely keep an eye on it. Thanks :)

[ Parent | on HN ]

Re:Overview of Text Extraction Algorithms by eli on Sunday March 20, 2011 @12:07AM (#2345495)

Which python port are you using? Last time I looked all the python Readability code I could find was either incomplete, old, or buggy.

[ Parent | on HN ]

Re:Overview of Text Extraction Algorithms by buss on Sunday March 20, 2011 @02:37AM (#2345706)

I've experimented with https://github.com/gfxmonk/python-readability but it's extremely slow. There's a decent fork called decruft that is a couple orders of magnitude faster http://www.minvolai.com/blog/decruft-arc90s-readability-in-p...

Decruft also has a couple bug fixes to python-readability. They both need a lot of work, though. You'll have to do some spelunking to figure out how to actually call the libraries correctly.

[ Parent | on HN ]