News for hackers. Stuff that matters.
Re:Typst: An easy to learn alternative for LaTex by mr-karan on Saturday July 20, 2024 @10:52AM (#41016888)
Typst has been pretty amazing, and at my organization, we’re very happy with it. We needed to generate over 1.5 million PDFs every night and experimented with various solutions—from Puppeteer for HTML to PDF conversions, to pdflatex and lualatex. Typst has been several orders of magnitude faster and has a lighter resource footprint. Also, templating the PDFs in LaTeX wasn’t a pleasant developer experience, but with Typst templates, it has been quite intuitive.

We’ve written more about this large-scale PDF generation stack in our blog here: https://zerodha.tech/blog/1-5-million-pdfs-in-25-minutes

Re:Typst: An easy to learn alternative for LaTex by happimess on Saturday July 20, 2024 @11:57AM (#41017384)
This is a really great write up. Kudos for the obvious effort, both on the technical side and sharing the process with the rest of us.
Re:Typst: An easy to learn alternative for LaTex by sitkack on Saturday July 20, 2024 @11:03AM (#41016967)
What is the use case for generating that many PDFs?
Re:Typst: An easy to learn alternative for LaTex by mr-karan on Saturday July 20, 2024 @11:07AM (#41016995)
Regulatory requirements mandate that. Stock brokers in India are required to generate this document called “Contract Notes” which includes all the trades done by the user on the stock exchanges. It also contains a breakdown of all charges incurred by the user (brokerage, various taxes etc). And this has to be emailed to every user before the next trading session begins.
Re:Typst: An easy to learn alternative for LaTex by bjourne on Saturday July 20, 2024 @11:22AM (#41017107)
There are of course way more efficient methods for generating templated pdfs than using a typesetter.
Re:Typst: An easy to learn alternative for LaTex by cameroncooper on Saturday July 20, 2024 @11:41AM (#41017235)
I'm interested to hear what you would propose.
Re:Typst: An easy to learn alternative for LaTex by abound on Saturday July 20, 2024 @01:15PM (#41017979)
Not sure what GP had in mind, but one can programmatically generate PDFs directly, without using something like Typst as a "middleman".
Re:Typst: An easy to learn alternative for LaTex by the_mitsuhiko on Saturday July 20, 2024 @01:40PM (#41018180)
Have you tried doing that? It’s no fun at all and far from easy. I don’t quite see a benefit in doing it without some utility.
Re:Typst: An easy to learn alternative for LaTex by a57721 on Saturday July 20, 2024 @04:50PM (#41019759)
Besides, the generation of PDF reports is usually decoupled from the templates, so you will have to work on your own "middleman".
Re:Typst: An easy to learn alternative for LaTex by wolfi1 on Saturday July 20, 2024 @01:16PM (#41017991)
I guess some webkit solution like wkhtmltopdf
Re:Typst: An easy to learn alternative for LaTex by mr_mitm on Saturday July 20, 2024 @01:30PM (#41018095)
How is that more efficient than Typst exactly?
Re:Typst: An easy to learn alternative for LaTex by bjourne on Saturday July 20, 2024 @01:32PM (#41018117)
Apache iText, for example.
Re:Typst: An easy to learn alternative for LaTex by IshKebab on Saturday July 20, 2024 @01:34PM (#41018134)
Does the law specify PDF? I would have thought pain text or even HTML would be sufficient.
Re:Typst: An easy to learn alternative for LaTex by the_mitsuhiko on Saturday July 20, 2024 @01:39PM (#41018173)
I don’t know the situation in India but brokers in Austria and Germany do the same. The law does not stipulate the format but PDF is what everyone uses. I assume it’s because it can be signed and archived and will outlast pretty much anything. You need to keep these for 7 years.
Re:Typst: An easy to learn alternative for LaTex by mr-karan on Saturday July 20, 2024 @02:11PM (#41018409)
Yes, in India, the law mandates that ECNs (electronic contract notes) need to be digitally signed with a valid certifying authority. While it's true that XML/docx/xls files could also support digital signatures, but I think PDFs are prevalent and also allow clients to verify this on their end, quite easily.
Re:Typst: An easy to learn alternative for LaTex by yukeabu on Saturday July 20, 2024 @04:48PM (#41019726)
PDF is less likely to contain executable malicious code than other formats.
Re:Typst: An easy to learn alternative for LaTex by radicality on Saturday July 20, 2024 @06:15PM (#41020507)
Is it? More so than say .csv file ?

I was under the impressions that pdfs are not that safe. I thought they can do stuff like execute a subset of PostScript and Javascript.

Re:Typst: An easy to learn alternative for LaTex by yukeabu on Saturday July 20, 2024 @07:21PM (#41020966)
Look, when it comes to corporate reporting, PDFs are pretty much the gold standard. Sure, they've got some potential security issues, but any decent company's IT department has them well in hand.

Think about it - you want your reports to look sharp, right? PDFs deliver that professional look every time, no matter who opens them or on what device. Plus, they've got all those nifty features like password protection and digital signatures that the big guys love.

CSV files? They're great for crunching numbers, but let's face it - they look about as exciting as a blank wall. Try sending a CSV report to the board of directors and watch their eyes glaze over.

So, yes, for reporting in a company that's got its security act together, PDFs are your best bet. They're like the well-dressed, security-savvy cousin of other file formats - they look good and keep things safe.

Re:Typst: An easy to learn alternative for LaTex by prmoustache on Sunday July 21, 2024 @04:10AM (#41023385)
More than plain text? I doubt so.
Re:Typst: An easy to learn alternative for LaTex by 1oooqooq on Sunday July 21, 2024 @05:42AM (#41023783)
common people don't talk about plain text. what are you? a hacker?!?
Re:Typst: An easy to learn alternative for LaTex by prmoustache on Sunday July 21, 2024 @08:32AM (#41024636)
I mean I guess you don't care as long as the file is signed if it is just some regulatory stuff that barely anyone would ever read anyway.
Re:Typst: An easy to learn alternative for LaTex by dudus on Saturday July 20, 2024 @04:23PM (#41019499)
Have you tried reportlab as well? It was a good solution when I had to deal with a similar problem many moons ago. Not quite the same volume you have but still.
Re:Typst: An easy to learn alternative for LaTex by kerkeslager on Saturday July 20, 2024 @04:54PM (#41019797)
Having used ReportLab a bunch, I'd agree it's a good solution, but not maybe on the more mediocre side of good. Generating LaTeX was a better solution for me, and while I haven't used it, Typst looks a lot better.
Re:Typst: An easy to learn alternative for LaTex by znpy on Saturday July 20, 2024 @06:03PM (#41020402)
Just wondering: did your organisation contribute anything back to the project, or supported it financially in any way?
Re:Typst: An easy to learn alternative for LaTex by mr-karan on Sunday July 21, 2024 @02:44AM (#41023007)
Hey. Yes we did support them financially from our FOSS fund. We’ll be happy to do it once more as well.
Re:Typst: An easy to learn alternative for LaTex by mstijak on Saturday July 20, 2024 @06:13PM (#41020479)
Never heard that someone is generating PDF documents at that pace. I'm working on a product that is used for mass PDF reporting based on Puppeteer. With nightly jobs, caching, and parallel processing, the performance is ok.

https://www.cx-reports.com

Re:Typst: An easy to learn alternative for LaTex by tecleandor on Sunday July 21, 2024 @05:21AM (#41023677)
I think we don't reach that much quantity, but we do a hefty number. What for? Invoices!

We're having problems because until now PDFs are being generated by the ERP system, and it can't keep pace. I know there's a Dev team working on a microservice for PDF generation, but never thinked about doing it with Typst.

I think I'm going to send them @mr-karan link.

Re:Typst: An easy to learn alternative for LaTex by 1oooqooq on Sunday July 21, 2024 @05:41AM (#41023773)
pdf/ps can easily be created in a way that data for text and qr code fields are easily in plain text. seems like yall focusing too much on the higher level tools instead of what's right in front of you.
Re:Typst: An easy to learn alternative for LaTex by mstijak on Sunday July 21, 2024 @07:21AM (#41024268)
Company branding is an important aspect of PDF creation that many tools struggle to handle correctly. PDF documents often need to include logos, company colors, fonts, and other branding elements. Puppeteer is popular because you can control these aspects through CSS. However, Puppeteer can be challenging to work with for larger documents, as each change requires programming effort or when your software needs to needs to serve multiple clients each with different requirements.
Re:Typst: An easy to learn alternative for LaTex by tecleandor on Sunday July 21, 2024 @10:35AM (#41025400)
Yep, it's mostly about branding and control. It needs certain concrete layout and logos, and has to be relatively easy to change them.

We also render shipping labels in PDF, and we have to be VERY strict with that. But we're still not touching that, as that process is not at slow and problematic as the invoicing one.

Re:Typst: An easy to learn alternative for LaTex by 1oooqooq on Sunday July 21, 2024 @11:03AM (#41025603)
how having the value and name in an invoice as plain text affect branding? or do you mean there client branding is added to the invoice??

the end result when you open the file is still a regular pdf. it's just encoded with some areas unpacked

Re:Typst: An easy to learn alternative for LaTex by tecleandor on Sunday July 21, 2024 @02:41PM (#41027155)
It's not about the plaintext, but about the layout, design and logos. At least in our case.
Re:Typst: An easy to learn alternative for LaTex by 1oooqooq on Sunday July 21, 2024 @04:18PM (#41027781)
so why you bring that up on a thread that had nothing to with layout? I'm honestly confused
Re:Typst: An easy to learn alternative for LaTex by iamgopal on Saturday July 20, 2024 @11:13PM (#41022094)
I was reading second sentence, and I knew it was zerodha. It’s good to see more open source in your tech stack.
Re:Typst: An easy to learn alternative for LaTex by stevekemp on Sunday July 21, 2024 @10:23AM (#41025303)
Thanks for your writeup, that was exceptionally well-presented.