Again, as far as I know, there are no heuristics good enough to get that right for all values of PDF.
The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a").
What? Why!? I've heard of doing that as a form of DRM, but I can't imagine Darwin defaulting to doing that.
This wouldn't be an issue if it was a conscious choice, but when I parsed a lot of born-digital PDFs we ended up with a lot that were like that from various source. Try explaining that...
See http://blog.idrsolutions.com/?s=%22Make+your+own+PDF+file for more info on how to write PDF by hand (also shows why I said you have to be stubborn to do that)
You are trying to move the goal post here. The above statement is simply untrue, PDF is not a text based format and it's that simple.
Look at Chapter 3, Syntax. The code is all text based. We are not talking about the visible characters in a PDF viewer, but the code of the PDF file itself.
BT /F13 12 Tf 288 720 Td (ABC) Tj ET
This can be extended to include spaces so you can essentially mark up entire lines of text at one time. What it can't do is cohesive paragraphs and flow/wrap, you need to use the relative positions to work out what text is in one block (and usually I'd defer to something like pdftotext for simple cases).
Laying out individual characters is common though. It's probably due to kerning concerns.
I silently thank every architect that provides searchable PDFs, it makes my job way easier
Look at Chapter 3, Syntax (and the rest of it, really).
I literally have scripts that use bash and sed, or Python, to modify PDFs by editing the text code. Doing it in Notepad is possible but tricky, as there's a table of object byte offsets near the end that it's easy to mess up by inserting a character.
As far as I know, any PDF can be losslessly converted to an equivalent PDF that can be edited in any text editor, even Notepad. And yes, you could fill in the forms there, too (if you were stubborn enough)