News for hackers. Stuff that matters.

A comment on What's so hard about PDF text extraction?. Parent comment

Re:What's so hard about PDF text extraction? by Alex3917 on Tuesday March 03, 2020 @03:06PM (#22478265)
Not really, it's just a different set of challenges. The original article sums it up well, in terms of a lack of text-order hints. I haven't really tried incorporating OCR approaches at all, but I suspect they could probably be used to detect hidden text.

The basic issue imho is that NLP algorithms are very inaccurate even with perfect input. E.g. even with perfect input, they're maybe only 75% accurate. And even an a text-processing algorithm that's like 99.9% accurate will yield input to your NLP algorithms that's like 50% accurate, so any results will be mostly unusable.

[ Parent | on HN ]
Re:What's so hard about PDF text extraction? by tastyminerals on Wednesday March 04, 2020 @01:35AM (#22481654)
NLP algorithms are just fine. It is the combination of regexes, NLP and deep learning that allows you to achieve good extraction results. So, basically OCR / pdf parser -> jpeg/xml/json -> regexes + NLP / DL extractor.
[ Parent | on HN ]