pkgwilliams

TeX used to have numerous unique features: fairly good fonts; automatic hyphenation, which makes it feasible to lay out text with full justification; support for non-English Latin characters, etc. The rest of the world has caught up with these.

IMO the ways in which it is still special are: (1) good math typography; (2) integration with automated bibliography systems; (3) easily generatable by other tools. (Lots of things that eventually turn into HTML hit #3, but not lots of things that eventually turn into really good-looking PDFs do.)

These points certainly aren't important in all cases ... but for what it's worth, arxiv.org hosts more than a million scientific papers, the vast majority of which are written in TeX.

The C/C++ code add up to about 120,000 lines.

Yes, it is massive and complex.

Fortunately, the core engine is essentially self-contained. Almost all of the things that make the TeX frustrating come from the UX, or the difficulty of adding new capabilities that call for somewhat more modern C code. By compiling all of the C with a Rust wrapper, we get a single binary out in the end, plus the ability to extend the engine in ways that were just not feasible before.

Revamping the core engine code from C to Rust would be nice, because it could be cleaner and more comprehensible, but it actually doesn't buy you anything in terms of the UX.

Edit: also, I didn't realize this when I made the decision to fork XeTeX, but LuaTeX has shifted to a C implementation that is more or less detached from the original WEB code.

Right.

The hope is that the easier development process will make it easier to add fancy features like microtypography ... but for me the real prize is HTML output that doesn't suck, so that's going to be the focus of my work.

I can't speak for latexmk, but Tectonic has, if I may say so myself, some quite sophisticated logic for managing reruns and figuring out which intermediate files can't be written to disk.

That being said, I wouldn't say that the `latexmk`-type functionality is the thing that is going to make Tectonic unique. Hopefully that will be the HTML output, but I have to implement that first ...

XeTeX accomplishes PDF output by piping through xdvipdfmx in the background, so it's the same process.