Text extraction

Posted by theslay on Saturday January 18, 2014 @08:44PM
from the self-post dept.

Hi, I'm working on plagiarism detection and I need some help on text extraction from pdfs. I've tried PDFTextStream which really works well for extracting text from pdfs. I need to be able to extract the text into a strutured format where i could query thing like title, chapters,etc. Would appreciate it if I could get pointers to achieving this task. Thanks

( 2 comments | on HN )

Re:Text extraction by pedalpete on Saturday January 18, 2014 @09:23PM (#7083147)

Have you tried posting this to http://stackoverflow.com ? That's a better forum for these kinds of questions.

If you were to write a blog post about how to structure the extracted text, that's more the HN thing.

[ Parent | on HN ]

Re:Text extraction by mindcrime on Saturday January 18, 2014 @09:52PM (#7083194)

I won't swear to it, but I suspect you're going to have to largely roll your own, and that it will be at least partly heuristic driven. I use Apache Tika[1] to extract text from PDFs and then index it with Lucene, but we don't need to discriminate between various chapters or anything. But I can picture how you could use OpenNLP[2] and some custom code, to break down the chapters.

[1]: http://tika.apache.org

[2]: http://opennlp.apache.org

[ Parent | on HN ]