Anyway, I don't remember exactly all the articles I tried, but two of them were
https://www.cs.princeton.edu/~chazelle/pubs/mst.pdf https://www.cs.ubc.ca/~condon/papers/chungcondon96.pdf
while one news that failed to parse was
http://www.repubblica.it/economia/2016/02/09/news/borse_9_fe...
I tried other articles and news, but I do not recall each of them exactly
The long version: It has to do with the underlying structure of the PDF; some of the characters in the above PDF have glyphs for display but don't actually map the characters to code points. So, when we pull out the text, they come through as invalid characters, which we should have filtered out. This is an issue we've seen with (all?) PDF viewers; the text you copy from a sentence isn't always what you expect... But, we're aware of that shortcoming and are looking at some ways to improve the quality.
In regard to the extra content in the news articles, we're not currently trying to do what BoilerPlate does. If you want to include or exclude specific content from a page, we have config options to do that via XPaths. Though, we're always open to ways of improving our services, and incorporating something like that would probably be useful.
Tika is able to parse them without a glitch, and has been doing so for a few years at least.
So I tried with some sample HTML taken from the main italian news sites. In this cases, it parsed it, but returned some garbage HTML tags and no content. BoilerPipe or similar libraries are able to extract a clean body of the news in all cases.
How is that state of the art technology?