As a part of our product development, we had fought with PDF so much, even we have a generic PDF parser with triple pipeline (One for single column, another for multi column and third for complex table based layouts) yet we are not getting 100% accuracy, I would say that it's bit risky to bet on PDF. PDF often is the most complex format ever made and it was never made for data extraction. And You are right that vision models are the only way but hallucination is real.