News for hackers. Stuff that matters.
Re:PDF Is the World's Most Important File Format by crazygringo on Friday May 03, 2019 @01:49PM (#19820586)
I didn't "get" PDF's until I started doing academic research.

But the ability to collect papers, books, documents, etc. all in a single format that I can read on any device, and mark up with highlights and notes, has been a game-changer.

Yes it's a lowest-common-denominator format. And it's designed for human reading and manual office tasks, not computer processing of data. But it works. And it's supported everywhere.

Doesn't matter if I'm on the Apple or Google or Microsoft or Adobe stack. Doesn't matter if the PDF is 20 years old. It just works.

Re:PDF Is the World's Most Important File Format by arcticbull on Friday May 03, 2019 @05:18PM (#19822372)
Some features, not all of them. Forms in particular are poorly supported in the Apple stack, and specifically, it can't generate QR codes from form fields in documents like Reader can. Fonts are usually inconsistent and wrong in forms. Letter spacing is all wrong. I'm sure there's tons of other things missing, the spec is ginormous and includes it's own version of JavaScript that's similar to -- but not quite -- ECMA which is a huge attack vector [0].

For instance, check out the Canadian passport simplified renewal form [1]. The upper-right corner on the first page is "$FORM$054(06-2018)$V$1.4$CS$0$C$0" on the Apple stack and a proper QR code which changes as you fill in the application in Reader. The big blue "Read Instructions" buttons don't work on the Apple stack either.

It may be important, but it's a waking nightmare of a spec.

[0] http://mariomalwareanalysis.blogspot.com/2012/02/how-to-embe...

[1] https://www.canada.ca/content/dam/ircc/migration/ircc/englis...

Re:PDF Is the World's Most Important File Format by mwfunk on Friday May 03, 2019 @08:14PM (#19823381)
Originally there were AcroForms, part of the PDF spec. That's supported by pretty much everyone.

Then, Adobe saw an opportunity make some enterprise money and introduced XFA (XML Forms Architecture). The XFA spec is associated with the PDF spec, and you can put XFA content inside of a PDF, but the XFA spec is actually larger than the PDF spec. It's utterly insane.

I know that Adobe of course supports XFA, and there are various enterprisey things that support it to varying degrees, but I don't think it's well-supported by anyone outside of Adobe's implementation. Not only is it huge and complicated, but it also requires pulling in a JavaScript interpreter, which is a big ask for a feature that only exists because at one time Adobe thought they could turn it into another revenue stream.

It's noteworthy that the PDF 2.0 spec specifically says that XFA is not only deprecated, but that any PDF which contains XFA content is considered out of spec for PDF 2.0. 2.0 goes back to AcroForms for all of that stuff and ditches XFA entirely. Likewise for JavaScript in general. Anything with JavaScript dependencies in PDF 1.7 is verboten in 2.0.

In general, if you get stuck with a PDF with XFA content (not AcroForms), your best bet is to just use Adobe Reader to fill it out. Hopefully PDF 2.0 will take over the world eventually and everybody will be 100% back on AcroForms.

Re:PDF Is the World's Most Important File Format by arcticbull on Friday May 03, 2019 @09:01PM (#19823585)
I'm surprised to hear they're getting rid of JavaScript (and I couldn't find any references to that, but I wish it to be true) -- but don't worry, they're adding a whole new pile of complexity in PRC. "PDF 2.0 adds support for “PRC”, a rich 3D modeling language, originally developed by Adobe." [1]

[1] https://theblog.adobe.com/taking-documents-to-the-next-level...

Re:PDF Is the World's Most Important File Format by Stratoscope on Friday May 03, 2019 @11:29PM (#19824245)
> I'm surprised to hear they're getting rid of JavaScript (and I couldn't find any references to that, but I wish it to be true)

I won't believe it until I see hard evidence. Acrobat and Reader have supported JavaScript for something like two decades. (I wrote a bunch of JavaScript multimedia APIs when I worked there around 2002.)

Consider the large number of interactive PDFs on the IRS website that use JavaScript for form calculations. I don't think Adobe is highly motivated to break all of those.

Re:PDF Is the World's Most Important File Format by blotter_paper on Saturday May 04, 2019 @01:43AM (#19824694)
I've seen some embedded SVGs get rendered differently by different PDF readers.
Re:PDF Is the World's Most Important File Format by sfkdjf9j3j on Friday May 03, 2019 @06:16PM (#19822751)
Not that you're wrong about the PDF spec being monstrously complex and not universally supported, but why would someone want to generate a QR code from form fields? Who actually wants to use QR codes?
Re:PDF Is the World's Most Important File Format by gibolt on Friday May 03, 2019 @08:40PM (#19823490)
Anyone in Asia, they are hugely popular and super useful.

Adding friends on WeChat, making payments to vendors, getting discounts, installing an app, etc

Re:PDF Is the World's Most Important File Format by geomark on Friday May 03, 2019 @09:26PM (#19823688)
Yep. In my case add friends on Line, buy subway tix, pay at 7-11 and many other places. You can also receive payments to your own QR code.
Re:PDF Is the World's Most Important File Format by Karunamon on Saturday May 04, 2019 @12:51AM (#19824535)
If I understand right, they took off pretty much everywhere other than NA for public use. The general vibe I get here is that nobody in the country uses them, and using them is seen as a bit of an anomaly, even scanning one in public makes you a minor spectacle.
Re:PDF Is the World's Most Important File Format by dd36 on Friday May 03, 2019 @06:31PM (#19822837)
A machine that scans forms.
Re:PDF Is the World's Most Important File Format by arcticbull on Friday May 03, 2019 @06:32PM (#19822843)
Haha, fair, that was just the first painpoint that sprung to mind. They're fairly common on visa and passport applications, and some other government forms. I guess if you're accepting paper submissions for whatever reason not having to transcribe the contents is a big win.
Re:PDF Is the World's Most Important File Format by nerdbaggy on Friday May 03, 2019 @11:17PM (#19824192)
QR codes are also super useful in warehouses
Re:PDF Is the World's Most Important File Format by wingerlang on Saturday May 04, 2019 @02:35AM (#19824817)
Like people said, QR codes are huge in Asia. I got one just yesterday to get my movie ticket.
Re:PDF Is the World's Most Important File Format by afiori on Saturday May 04, 2019 @07:26AM (#19825802)
> Forms in particular are poorly supported

I understand this is a problem as it is one extremely important features of PDFs, but from a consumer perspective (never worked in an big office) I do not think I have ever seen a PDF form.

Maybe the real success of PDFs is that it managed to hide all the inner complexity of the format and missing features from the common use cases.

Re:PDF Is the World's Most Important File Format by sbr464 on Saturday May 04, 2019 @03:16AM (#19824965)
re: forms, I find it easier to just convert a pdf to an image or screenshot it programmatically, and process it with a few custom OCR APIs that identify key/value form structures. Unfortunately, I would say, but it works pretty well.
Re:PDF Is the World's Most Important File Format by mikeash on Friday May 03, 2019 @04:39PM (#19822058)
I wish it really Just Worked everywhere. Unfortunately, the fixed layout and tendency to target 8.5x11 means they’re basically unreadable on smaller screens.
Re:PDF Is the World's Most Important File Format by username223 on Friday May 03, 2019 @11:10PM (#19824153)
Print or screen: pick one.

If you want something to print beautifully, you take the time to put the figures and pictures exactly where you think they fit. If you want something to be read with arbitrary text sizes, you hope they'll show up somewhere near the relevant text. Treating a book as an ebook, or vice versa, will always be a compromise.

Re:PDF Is the World's Most Important File Format by mikeash on Saturday May 04, 2019 @11:11AM (#19826907)
Easy choice: screen.
Re:PDF Is the World's Most Important File Format by username223 on Saturday May 04, 2019 @12:57PM (#19827658)
Horses for courses. For novels? Sure. For reference books with a lot of photos and diagrams, I still prefer print to whatever would result from a "reactive" layout reacting to my current screen and font size. Maybe they could be rewritten as interactive "apps," but that's a lot more work, opening a whole new can of worms.
Re:PDF Is the World's Most Important File Format by azhenley on Friday May 03, 2019 @07:36PM (#19823205)
The new ACM format is attempting to fix this for academic publications. I believe it generates HTML and should look better across devices (and be more accessible).

I haven't seen that happen yet though...

Re:PDF Is the World's Most Important File Format by SiVal on Friday May 03, 2019 @08:24PM (#19823423)
I haven't heard of this "new ACM format", but for years I've wished for a PDF2 that incorporated lessons learned from PDF and web pages for print, ebooks, web on multiple screen sizes, and non-visual readers (for the blind but also for anyone who would like to "read" a book or paper while driving or jogging).

A "multi-view" format that could be viewed as full page layout (both scrolling and paged), small screen linear, audio-only linear, or machine-readable (full, deterministic text sequence, tabular data, alt-text and tags for images and charts, etc.) All pages, images, fonts, etc., would remain encapsulated within a single file.

The closest I've seen is the latest ebook format, but that doesn't have anything close to the power of PDF to lay out a proper textbook or magazine article page.

Re:PDF Is the World's Most Important File Format by watchdogtimer on Friday May 03, 2019 @09:18PM (#19823660)
Isn't that just HTML?
Re:PDF Is the World's Most Important File Format by dzhiurgis on Saturday May 04, 2019 @01:27AM (#19824638)
Markdown comes to mind
Re:PDF Is the World's Most Important File Format by jlarocco on Friday May 03, 2019 @07:22PM (#19823139)
That's true, but not unique to PDF. Any longer document is "basically unreadable on smaller screens".
Re:PDF Is the World's Most Important File Format by mikeash on Friday May 03, 2019 @07:37PM (#19823209)
No way. A Kindle is great for reading books, but text in a typical PDF is too small to read. Reading on a phone is a bit more annoying but totally reasonable with dynamic text layout.
Re:PDF Is the World's Most Important File Format by mwfunk on Friday May 03, 2019 @09:19PM (#19823663)
Sadly PDFs are usually just a series of graphics instructions for drawing stuff (including text) at specific sizes and positions, on a page with a defined size. It's an inherently hostile format for reflowing text, although I think some PDF viewers can do that to an extent. The PDF accessibility spec defines an optional, more DOM-like document structure that a compliant viewer could use to reflow text, but in my experience even the apps that produce that structure don't populate it with enough data to make reflowing reliable or easy to implement. Pretty much every format is way better for reflowing text than PDF (Kindle, ePub, HTML, etc.).
Re:PDF Is the World's Most Important File Format by marvindanig on Friday May 03, 2019 @10:25PM (#19823944)
+1 also, kindle != web.
Re:PDF Is the World's Most Important File Format by _emacsomancer_ on Friday May 03, 2019 @09:02PM (#19823588)
Koreader[0] can be sideloaded on Kindles, Kobos and other devices, and is good at making PDFs readable on such devices.

[0] http://koreader.rocks/

Re:PDF Is the World's Most Important File Format by mikeash on Saturday May 04, 2019 @11:12AM (#19826911)
Thanks, I’ll have to check this out.
Re:PDF Is the World's Most Important File Format by diffeomorphism on Friday May 03, 2019 @05:16PM (#19822356)
> and mark up with highlights and notes, has been a game-changer.

What are you using for that? In my experience annotation support is very spotty, not user-friendly and not cross-platform, so I rarely use it for collaboration. Even fill-in forms one gets for reimbursements often seem badly done with too small or too large boxes or missing boxes. Thus, pdf seems fine for read-only, but making changes afterwards is not a good experience.

Re:PDF Is the World's Most Important File Format by 3xblah on Friday May 03, 2019 @10:38PM (#19824006)
I could say all the same things about text files or JPGs.

PDF is not the "lowest common denominator".

That would be the format that can be most easily converted to other formats.

With text and images, I can easily create PDFs and documents in myriad other formats.

Alas, PDFs do not convert as easily to other formats. To this day, no one has a PDF-to-text converter that is 100% reliable in preserving the proper line lengths. Not even a company with Google's resources.

Re:PDF Is the World's Most Important File Format by rowanG077 on Saturday May 04, 2019 @11:37AM (#19827102)
Doesn't HTML also provide this feature? While being vastly more powerful and the only downside it cannot be printed accurately?
Re:PDF Is the World's Most Important File Format by nikofeyn on Friday May 03, 2019 @03:58PM (#19821725)
it would be much better if the djvu format, wholly superior to pdf for large files, wasn't locked behind a license.
Re:PDF Is the World's Most Important File Format by jonathanyc on Friday May 03, 2019 @04:06PM (#19821780)
PDF has extremely developed vector graphics support as well as adequate compressed image support (e.g. you can compress using a bunch of methods as good as what DJVU uses for your purpose, including PNG and JPEG2000.)

AFAIK DJVU is a much simpler format primarily for scanned images, not vector graphics (which includes LaTeX-generated documents, mind you.) Could anyone with more knowledge about DJVU comment?

Re:PDF Is the World's Most Important File Format by jwilk on Friday May 03, 2019 @04:50PM (#19822145)
Yes, DjVu has raster images under the hood.
Re:PDF Is the World's Most Important File Format by jwilk on Friday May 03, 2019 @04:49PM (#19822136)
What do you mean by "locked behind a license"?
Re:PDF Is the World's Most Important File Format by hcs on Friday May 03, 2019 @10:02PM (#19823837)
There are a few patents involved: http://djvu.sourceforge.net/licensing.html
Re:PDF Is the World's Most Important File Format by swiley on Friday May 03, 2019 @11:02PM (#19824108)
HTML (sans javscript) would have been so much nicer though. All of the machine readable structure and separation of style from content would have been just absolutely wonderful.

But we'll never have that (not in the sense we have PDF.) Either it's an image or an application, no one really understands how to handle anything in between.

Re:PDF Is the World's Most Important File Format by aabbcc1241 on Sunday May 05, 2019 @01:43AM (#19831364)
And you can get better accessibility with html: reflow, adjust font size, adjust color, e.t.c. I only found a few PDF reader can display the content well on the mobile phone, but still worse experience when compare to the web.
[dead]
Re:PDF Is the World's Most Important File Format by diplocorp on Friday May 03, 2019 @10:35PM (#19823990)
Have you set up a bot that copy-pastes this every time 'PDF' is a substring of a post's title? Seriously, stop spamming.
Re:PDF Is the World's Most Important File Format by yjftsjthsd-h on Friday May 03, 2019 @09:06PM (#19823609)
> and spaced repetition via Anki.

Now that is a potentially killer feature. Anki is great once you get information in, but in my mind it always laggnd the proprietary one... Supermemo?.. in terms of getting stuff into cards and your brain in the first place.

Re:PDF Is the World's Most Important File Format by blairbeckwith on Friday May 03, 2019 @06:48PM (#19822948)
Man, this is so annoying.
Re:PDF Is the World's Most Important File Format by Sabinus on Friday May 03, 2019 @07:27PM (#19823161)
He's not spamming, he made it himself, and I could really see it helping people.
Re:PDF Is the World's Most Important File Format by llao on Saturday May 04, 2019 @03:51AM (#19825063)
Then why would they hijack the top comment about widespread PDF support instead of making a separate top-level post when the whole submission is about PDF?
Re:PDF Is the World's Most Important File Format by mbreese on Friday May 03, 2019 @07:19PM (#19823124)
In what way? It seems interesting to me.
Re:PDF Is the World's Most Important File Format by mkl on Friday May 03, 2019 @08:36PM (#19823470)
Yes, it's neat and interesting, but burtonator posts basically the same comment in almost every thread about PDFs, academic papers, etc.
Re:PDF Is the World's Most Important File Format by whenchamenia on Friday May 03, 2019 @02:07PM (#19820763)
Except for many, it doesen't. Why glorify a closed format that we spent decades trying to aviod just because its common now? The inherent issues persist.
Re:PDF Is the World's Most Important File Format by starsinspace on Friday May 03, 2019 @02:17PM (#19820866)
PDF has been released as an open standard since 2008.
Re:PDF Is the World's Most Important File Format by thro_away_n on Friday May 03, 2019 @02:26PM (#19820952)
198 CHF to obtain ISO 32000-2:2017 from ISO, however.
Re:PDF Is the World's Most Important File Format by Arkanosis on Friday May 03, 2019 @04:44PM (#19822095)
That's the same price as C++ (ISO/IEC 14882:2017) and much cheaper than the entire languages codes standard (ISO 639-1:2002, ISO 639-2:1998, ISO 639-3:2007, ISO 639-4:2010 and ISO 639-5:2008).

I get your point, though.

Re:PDF Is the World's Most Important File Format by boshomi on Friday May 03, 2019 @02:16PM (#19820853)
since ISO 32000-2 there is no proprietary technology included[1]

[1] https://www.iso.org/standard/63534.html

Re:PDF Is the World's Most Important File Format by throwaway2048 on Friday May 03, 2019 @02:41PM (#19821089)
Not all PDFs are standards compliant, plenty are filled with acrobat only JS, embedded flash and form filling crap.
Re:PDF Is the World's Most Important File Format by semi-extrinsic on Friday May 03, 2019 @03:39PM (#19821562)
As e.g. the fine men and women of Int. J. POC||GTFO have demonstrated, it's possible to embed all sorts of weird and wonderful things in a PDF. I think a format where it's possible to include as much as possible is better.

Just look at HTML. Would you rather have it be impossible to extend beyond text, images and stylesheets?

Re:PDF Is the World's Most Important File Format by na85 on Friday May 03, 2019 @07:45PM (#19823256)
I would. The modern web is a rich-get-richer hellscape of surveillance and advertising.
Re:PDF Is the World's Most Important File Format by icxa on Friday May 03, 2019 @06:38PM (#19822881)
Form filling is part of the standard, specifically, the process of applying an XFDF to a PDF, which even allows you to do things like present a remote web form and then inject the data into the PDF.
Re:PDF Is the World's Most Important File Format by jimjimjim on Friday May 03, 2019 @05:04PM (#19822260)
html can include acrobat only JS and embedded flash and form filling crap.