News for hackers. Stuff that matters.
Re:Show HN: Talk to any ArXiv paper just by changing the URL by pugio on Thursday December 21, 2023 @12:50AM (#38717394)
I've spent the last couple weeks diving into various PDF parsing solutions for scientific documents. GROBID is pretty cool, but it made some mistakes when trying to parse (I think arxiv) papers which removed some of the text.

Even though it gave a lot of great structured options, missing even a single sentence was unforgivable to me. I went with Nougat instead, for arxiv papers.

(Also check out Marker (mentioned on hn in the last month) for pretty high fidelity paper conversion to markdown. Does reasonable job with equations too.)

Re:Show HN: Talk to any ArXiv paper just by changing the URL by skeptrune on Thursday December 21, 2023 @01:34AM (#38717679)
Did you try Apache Tika?
Re:Show HN: Talk to any ArXiv paper just by changing the URL by kordlessagain on Thursday December 21, 2023 @10:44AM (#38721881)
Google's Document AI does a good job, but I'll need to test the equation handling again to be sure.