News for hackers. Stuff that matters.

Show HN: CLI for generating PDFs for offline reading

https://github.com/dvcoolarun/web2pdf

I've always thought that extensive reading was best suited for the realm of paper. As a result, I've created a command-line interface (CLI) tailored for my own use and decided to make it open source. I welcome any feedback you may have.

[Edit] Sample PDF :: https://drive.google.com/file/d/1n7M1TKOptSsYiibrbvV_Yojx53T...

( 43 comments | 50 points | on HN )
Re:Show HN: CLI for generating PDFs for offline reading by ComputerGuru on Monday February 05, 2024 @02:50PM (#39266128)
I feel like if you are claiming "beautiful" output then it's obligatory to have at the very least screenshots of said output PDFs (or better yet, a sample for the same link in the CLI screenshot, especially so people can see how the text flows, what quality images are captured at, how text can be selected, etc).
Re:Show HN: CLI for generating PDFs for offline reading by dvcoolarun on Monday February 05, 2024 @03:47PM (#39266931)
Arr! Just updated the post with a sample PDF.
Re:Show HN: CLI for generating PDFs for offline reading by Hamuko on Monday February 05, 2024 @05:00PM (#39267840)
Is it just me or does the sample not look that beautiful?
Re:Show HN: CLI for generating PDFs for offline reading by doctorwho42 on Monday February 05, 2024 @07:34PM (#39269221)
Kinda looks like the script just grabbed the <p>'s and did a copy paste. Without much thought to format
Re:Show HN: CLI for generating PDFs for offline reading by tomcam on Monday February 05, 2024 @07:47PM (#39269321)
Correct, it is only you who looks that beautiful
Re:Show HN: CLI for generating PDFs for offline reading by dheera on Monday February 05, 2024 @05:04PM (#39267891)
I'd strongly suggest to include the sample PDF screenshot in the Github readme as well.

Also just my opinion and honest feedback but I don't really consider that PDF "beautiful" or readable. However, your project is a very good start toward something that is potentially very useful.

Here are my suggestions:

- Switch to a modern, beautiful, readable serif font that isn't an overused (Times New Roman, Garamond, and pretty much any MS fonts are overused and will make your PDF end up looking like "yet another Word document"). Some good options that come to mind are Tiempos Fine, Butler, Lora, but there are lots of good ones out there. Shop around for something both unique AND highly readable. Make sure your PDF generator embeds the font or (preferably) converts all text to paths instead of just referencing the font.

Also, know the difference between fonts designed for printing and screen reading and fonts designed for both, and adjust according to whatever use case you're marketing.

- Figure out how to make the sub-headers heavier weight and significantly larger to stand out. Be bold with your font size changes. Printed documents look "nice" when the sections are clearly laid out and EASY for the eye to parse from a distance how the document is organized and which are top level headings and which are subheadings under those headings. For generation purposes there's a good chance you can probably just parse it from <h1> <h2> <h3> ... tags in the HTML, unless the website uses some JS-framework-of-the-month bullshit. Use more white space before each header to physically separate it. Avoid using lines between a header and the text it corresponds to, as that can be distracting to the eye since lines are normally thought of as separating sections.

I'd give you a pull request but I'm super busy these days.

Re:Show HN: CLI for generating PDFs for offline reading by starbugs on Monday February 05, 2024 @05:49PM (#39268304)
Just one thing to add: Reduce the line length and make the font slightly bigger.

Maybe increase the line height a bit.

Re:Show HN: CLI for generating PDFs for offline reading by algon33 on Monday February 05, 2024 @02:53PM (#39266174)
Yeah, that would make this post a lot better.
Re:Show HN: CLI for generating PDFs for offline reading by abhibeckert on Monday February 05, 2024 @07:03PM (#39268947)
Agreed. I would offer a PDF version of the project's readme file as a demo.

OP: this is important. There are a million tools to generate PDF files, most of them don't produce nice looking PDFs.

Re:Show HN: CLI for generating PDFs for offline reading by jackconsidine on Monday February 05, 2024 @03:24PM (#39266606)
This is cool! I have a HN pipeline where I upvote things that I want to drill into, and a script I wrote generates PDFs and sends to my Kindle for offline reading (great for my pipeline). That uses Playwright's "to PDF" method which is over the browser and slow. I might look into replacing with this.

If there's any interest I might OSS the pipeline

Re:Show HN: CLI for generating PDFs for offline reading by Jun8 on Monday February 05, 2024 @03:30PM (#39266710)
This sounds like an excellent idea!
Re:Show HN: CLI for generating PDFs for offline reading by unfamiliar on Tuesday February 06, 2024 @02:08AM (#39271632)
PDF sounds like a terrible format for reading on the kindle, why not MOBI or epub or whatever it uses?
Re:Show HN: CLI for generating PDFs for offline reading by majora2007 on Tuesday February 06, 2024 @03:34PM (#39280180)
I was thinking the same thing. Why wouldn't this project convert to epub which is easier to pack and much easier to read?
Re:Show HN: CLI for generating PDFs for offline reading by worldsayshi on Tuesday February 06, 2024 @05:17PM (#39281545)
I wonder if Pandoc could do this well.

Edit: Ah, there was a thread(1) with related examples further down. And Pandoc can indeed do epub output.

1 - https://news.ycombinator.com/item?id=39268147

Re:Show HN: CLI for generating PDFs for offline reading by dv35z on Tuesday February 06, 2024 @11:34AM (#39276437)
Could you share more details about this? The tools / flow?
Re:Show HN: CLI for generating PDFs for offline reading by jll29 on Monday February 05, 2024 @03:59PM (#39267084)

  % python main.py https://www.paulgraham.com/avg.html
  Traceback (most recent call last):
  File "/Users/bill/web2pdf/main.py", line 7, in <module>
    from readability import Document
  ImportError: cannot import name 'Document' from 'readability' 
  (/Users/bill/.local/share/virtualenvs/web2pdf- 
 gXeVRXKg/lib/python3.9/site-packages/readability/__init__.py)
But according to your Pipfile.lock, the readability module needed is 0.3.1:

  "readability": {
            "hashes": [
              "sha256:f9030df8bc31aad45baffa9a2d9ce1fdd8051833e5b5bda3027df32fdec00fad"
            ],
            "index": "pypi",
            "version": "==0.3.1"
        },
Version 0.3.1 of the module "readability" exists, but does not appear to have a class "Document".
Re:Show HN: CLI for generating PDFs for offline reading by pbourke on Monday February 05, 2024 @04:07PM (#39267195)
I think the pipfile should specify only the ‘readability-lxml’ package and not ‘readibility’, which does unrelated things.
Re:Show HN: CLI for generating PDFs for offline reading by dvcoolarun on Monday February 05, 2024 @02:59PM (#39266270)
Apologies for the oversight; I forgot to include the screenshot of the sample PDF. Here it is for your reference: https://drive.google.com/file/d/1n7M1TKOptSsYiibrbvV_Yojx53T...
Re:Show HN: CLI for generating PDFs for offline reading by fdgadfagfgd on Monday February 05, 2024 @03:04PM (#39266349)
What was the website source?
Re:Show HN: CLI for generating PDFs for offline reading by fdupress on Monday February 05, 2024 @03:16PM (#39266511)
Re:Show HN: CLI for generating PDFs for offline reading by seabass-labrax on Monday February 05, 2024 @02:30PM (#39265838)
Very interesting! One piece of feedback: it would probably be more useful to have a screenshot of the PDF on your README rather than one of the CLI. Also, do you intend to release this as FOSS?
Re:Show HN: CLI for generating PDFs for offline reading by jll29 on Monday February 05, 2024 @03:56PM (#39267035)
Have you compared it with a conversion by pandoc (https://pandoc.org/)?
Re:Show HN: CLI for generating PDFs for offline reading by bomewish on Tuesday February 06, 2024 @03:16AM (#39272104)
Came for this. Whole thing would look nicer by just using markdownload -> pandoc. This is a solved problem and the existing solution seems more elegant. It’s cool to do projects tho.
Re:Show HN: CLI for generating PDFs for offline reading by Throw73747 on Monday February 05, 2024 @03:31PM (#39266724)
Parhaps add ublock filters support? I use it to strip down any unwanted elements on page before printing. On hacker news discussions it removes forms, reply links, header and footers...
Re:Show HN: CLI for generating PDFs for offline reading by OhMeadhbh on Monday February 05, 2024 @03:21PM (#39266569)
Apropos of nothing, I added this function so I don't have to leave the command line to see the PDF.

   pdfpage() {
     convert -resize 0x1000^ "${1}"[${2}] -background white -flatten sixel:-
   }
You can probably deduce it assumes you have a Imagemagick installed and you're in a terminal with sixel support.
Re:Show HN: CLI for generating PDFs for offline reading by skanga on Monday February 05, 2024 @03:08PM (#39266402)
Found some potential bugs. Please check the github issues page.
Re:Show HN: CLI for generating PDFs for offline reading by K2h on Monday February 05, 2024 @03:00PM (#39266287)
Very cool! in README.md is that an extra 'p' in Webp2pdf ?
Re:Show HN: CLI for generating PDFs for offline reading by Someone on Monday February 05, 2024 @05:04PM (#39267884)
FTA: “Then you can use the tool as follows

  pipenv shell
  pipenv install
  python main.py https://www.paulgraham.com/avg.html, https://www.paulgraham.com/determination.html
Just add the webpage URLs separated by commas”

What’s the rationale for “separated by commas”? The convention for CLI arguments is to use one argument per input file.

Re:Show HN: CLI for generating PDFs for offline reading by dheera on Monday February 05, 2024 @05:11PM (#39267952)
Also commas are valid characters in an URL, so they should not be used as a separator for URLs.

If you want to support multiple input URLs just pass all of them in, one URL per element in sys.argv

Re:Show HN: CLI for generating PDFs for offline reading by pavs on Monday February 05, 2024 @05:30PM (#39268147)
like this:

  sudo apt install pandoc wkhtmltopdf
  npm install -g readability-cli
  pandoc -s https://www.paulgraham.com/avg.html -o output.html && readable output.html -o readable.html && wkhtmltopdf readable.html output.pdf &&  open output.pdf
going even further using bash script to prompt for url.

  #!/bin/bash

  # Prompt the user for a URL
  read -p "Enter the URL: " URL

  # Use the URL in the pandoc command
  pandoc -s $url -o output.html && readable output.html -o readable.html && wkhtmltopdf readable.html output.pdf && open output.pdf


  chmod +x web2pdf.sh
  # add an alias to bashrc
  alias web2pdf='/path/to/your/web2pdf.sh'
  source ~/.bashrc
Re:Show HN: CLI for generating PDFs for offline reading by natrys on Monday February 05, 2024 @06:31PM (#39268676)
Pandoc can also output to pdf if you have latex installed. You can use tectonic if you want a single binary solution than a full latex toolchain, it downloads necessary packages on the fly.
Re:Show HN: CLI for generating PDFs for offline reading by psanford on Monday February 05, 2024 @06:10PM (#39268501)
wkhtmltopdf seems to be archived on github[1]. Can't say I want to run an unpatched, outdated browser engine for this.

[1]: https://github.com/wkhtmltopdf/wkhtmltopdf

Re:Show HN: CLI for generating PDFs for offline reading by bananskalhalk on Tuesday February 06, 2024 @12:51AM (#39271230)
Or you could just import weasyprint and readability in python!

Which is pretty much what OP did, with a few nice additions like nice output and some custom css and fonts. Which is nice and OP hope brings value to people.

If you add a few printf:s and include a nice template for pandoc* it would be more along the same thing.

*) Neither pandoc, wkhtmlpdf or weasyprint have (in my opinion) bad default templates.

Re:Show HN: CLI for generating PDFs for offline reading by worldsayshi on Tuesday February 06, 2024 @05:22PM (#39281609)
Pandoc can also do epub output: https://pandoc.org/epub.html
Re:Show HN: CLI for generating PDFs for offline reading by harry8 on Monday February 05, 2024 @06:58PM (#39268902)
Webbrowser opens url -> print -> save as/to pdf?

I'm sure I'm missing something, what is a cli interface buying me here?

Re:Show HN: CLI for generating PDFs for offline reading by rahimnathwani on Monday February 05, 2024 @08:25PM (#39269617)
For print or PDF, I like multi-column newspaper style, as created by this extension: https://chromewebstore.google.com/detail/simple-print/nalmbm...

One benefit of using a Chrome extension (vs. CLI) is that it's easy to 'print' things that require authentication.

Re:Show HN: CLI for generating PDFs for offline reading by codeonline on Monday February 05, 2024 @09:23PM (#39270002)
Can you add comparison pdfs generated by pandoc and gotenberg?
Re:Show HN: CLI for generating PDFs for offline reading by dvcoolarun on Monday February 05, 2024 @10:22PM (#39270391)
Arr, this blew up! I think, in some form, people are missing the context of the script. It's a plug-and-play script where you can make changes to PDF quality using CSS/Python. Even fonts are loaded through Google in Python. 'Beautiful' is called contextual. You can create your own version and share it with the community.

I'm on mobile, so I can't add a Google Drive file screenshot to the readme, and iframes are not supported.

Re:Show HN: CLI for generating PDFs for offline reading by fishywang on Tuesday February 06, 2024 @12:47AM (#39271216)
Somewhat similarly, I wrote a web app to generate epub (instead of pdf) out of urls and send to eink reader(s) directly (via a telegram bot) so I can read them. Currently it supports sending epub by email (for kindle) or uploading epub to dropbox (for kobo, etc.). It originally also supports reMarkable cloud but we can no longer make reMarkable cloud actually work. There's also a REST api to generate epub to be downloaded directly: https://github.com/fishy/url2epub/blob/main/REST.md

For e-ink readers epubs are generally better than PDFs for urls anyways, as epubs are basically packed htmls, and also the flow text works better on smaller screens.

Re:Show HN: CLI for generating PDFs for offline reading by sn0n on Tuesday February 06, 2024 @02:27AM (#39271780)
Does it run a headless chrome for pixel perfect formatting as laid out as a webpage and applied in that format to PDF ignoring the pages print css rules? Cuz, that would be a nice start. And an option for size to be pixel width based for ideal layout... Because I won't be printing, I will be viewing on my phone, so one overly large page would be perfect.
Re:Show HN: CLI for generating PDFs for offline reading by adrian_b on Tuesday February 06, 2024 @06:11AM (#39273084)
Both Chrome and Firefox have absolutely horrible "Print" (to PDF) commands, which render the Web pages in a different way than what they show on the screen, and which results in large parts of the page being obscured by ads, menus, headers, etc., or in parts of the Web page that are outside the rendered area, so they are missing, or in content that is compressed to a small part of the output pages.

It would be really nice if there existed a utility able to produce a PDF file where the Web pages are rendered as well as the browsers render them on the screen, without becoming confused even by complex scripts loaded by the page.

The alternatives to "Print" (producing a PDF) are even worse. A screenshot has limited resolution and it loses the text. In the past "Save as ..." was the normal solution, but now even if you save a "complete" page, it will still frequently include scripts that will no longer work offline. What I want to save are the pages perfectly rendered as they were at that instant, without any scripts that could make them appear differently in the future.

Re:Show HN: CLI for generating PDFs for offline reading by gildas on Tuesday February 06, 2024 @07:55AM (#39273684)
You should try SingleFile, see https://github.com/gildas-lormeau/SingleFile
Re:Show HN: CLI for generating PDFs for offline reading by nacho2sweet on Tuesday February 06, 2024 @12:16PM (#39277171)
We just use a headless chrome with a sort of wrapper script to do this at my work with a bunch of settings close to the actual size of paper. It allows me to test all of our reports in media->print in dev tools then print->pdf with chrome and only have to design to that spec. Then in our reports we provide a "save as pdf" button instead of encouraging print in all the other possible browsers which would make the task insane and cause me to possibly quit.