"[I]t already broke backwards compatibility several times". There's your answer. I've got three decades worth of style files, personal macros, and workflow invested in the way I use LaTeX, numbering in the thousands of lines. In some communities, especially academia and most especially amongst mathematicians and engineers, that is a typical investment. Nothing that might require me to start overhauling that is going to get a moment's consideration until LaTeX stops working for me, and even then I'm probably going to try and maintain a bespoke TeX distro before I'd try to replace it. New and casual users of LaTeX should probably consider these alternatives (I've even considered moving over to LuaTeX because it would allow more automation, but I'd need a pretty longish period of reimplementation that I don't see becoming available soon.) But anyone who wants to move that community of long-time users, organizations and publications is going to need to address the problem of a significant user base that have been using effectively the same software for literally (and according to my students, unbelievably) decades.

There's an analogous situation (or a continuation of this same situation, given the overlap in user base) for those who've invested decades in building a personal set of Emacs configuration files. Again, I've got thousands of lines of *.el files (many of them devoted to my idiosyncratic use of LaTeX) in my Emacs configuration. Of projects looking to replace the aged Elisp engine and create a modern implementation aren't going to get much consideration if it would mean replacing that (literally decades of) work in a different language. That's why I continue to be interested in Guile Emacs---less because of Guile (although I have no issues replacing Elisp with a modern Scheme), but because they've always made it clear that an ability to re-use Elisp configuration files was a primary design goal.

That's exactly it - backwards compatibility. Back at the end of the eighties I was desperately trying to find a way to update documents on-site when I visited customers (and I would sometimes stay for weeks or months), where I couldn't use the word processing system we used at home. I needed something I could run locally, on the laptop (yes we had them) as well as on the target systems (servers). Tried a lot of stuff.. eventually found LaTeX, wrote software to convert our old documents to LaTeX, and a document class which created the same layout as the original (most LaTeX converters "hardcode" the layout [or at least used to, back then] - won't do, as my company changes the "template" now and then). Everything finally ok. I and colleagues could write documents on-site. A guy from another company had the same problem so I wrote a document class for his company's layout as well. So, I have these documents from way back, in LaTeX, and occasionally we have to produce them again. Sometimes we re-create them with a newer document class. Other times we extract chapters to be included in newer documents. If LaTeX had broken backwards compatibility it would be disastrous. I won't touch any replacement which doesn't guarantee backwards compatibility. However old the documents are.

> (I've even considered moving over to LuaTeX because it would allow more automation, but I'd need a pretty longish period of reimplementation that I don't see becoming available soon.)

Good thing you didn't bother, because it's absolutely not worth it. I chose LuaTeX initially precisely because it seemed more convenient to write commands in Lua than in plain LaTeX, but it turned into a total nightmare. Issues crop up as soon as you try to do something with text that embeds LaTeX macros, and the errors you get are even more byzantine than the ones you'd get by writing "normal" LaTeX code.

IMHO the very idea of embedding an external language into LaTeX is pointless, for the simple reason that LaTeX isn't markup, but code, and that it's pretty much impossible to parse it into a representation that makes sense and could be manipulated like, say, the HTML DOM. The only sane way to automate things is to write your documents in some language or markup that can be compiled to LaTeX, or to use a template engine.

> Good thing you didn't bother, because it's absolutely not worth it.

The biggest thing LuaTeX gives you is variou hooks and access to various data structures used internally by the typesetting engine, so that you can work with them directly instead of having to do everything in a roundabout way through macro expansion. If all you need is text expansion then sure, just use TeX/LaTeX macros. (You mentioned "try to do something with text that embeds LaTeX macros"; I think working at the text level like that is a sign that your problem, or at least your solution, may not be a good fit for what LuaTeX gives.)

Here are a couple of my answers where LuaTeX was valuable; you can find hundreds more (IMO) by others:

- https://tex.stackexchange.com/a/379802 Avoid getting short words at line edges: Note that the macro-level solution runs into issues with \ref etc (just as you said about text that embed LaTeX macros), but the LuaTeX solution works perfectly in all cases

- https://tex.stackexchange.com/a/401315 Letter-spacing

- https://tex.stackexchange.com/a/403353 Typesetting on a poster with some image cut out

- https://tex.stackexchange.com/a/416772 Doing text replacements based on TeX macro state (maybe this could be done without LuaTeX?)

- https://tex.stackexchange.com/a/414013 Inserting random kerning

I do completely agree with your recommendation of generating TeX/LaTeX if possible (and keeping the macros/LaTeX to a minimum).

ConTeXt is not targeted toward authors of scientific papers (although I'm sure this is possible). The community behind ConTeXt has always been involved with books and more complex material. So I guess they evolved into a niche of the TeX market, like music typesetting (MusiXTeX and friends).

Regarding Lua, I think it is a strange idea from the point of view of the typical LaTeX package writer. Not only you need to learn Lua, but you also have to write packages that are accepted only by this implementation of LaTeX. This is a clear disadvantage if you want to have a widely used package.

> ConTeXt is not targeted toward authors of scientific papers

Absolutely true.

I would posit that most people writing LaTeX aren't writing up papers in it either (Most journals prefer or require Word these days).

ConText isn't just for books though..it's great for any sort of thing with formatting...like, say, a resume.

>I would posit that most people writing LaTeX aren't writing up papers in it either

what exactly do you think they're writing in it then? business cards?

as the other commenter said: publishing across math/physics/cs i have never seen a journal not prefer latex to word. and while ieee accepts word it most certainly prefers latex:

https://www.overleaf.com/org/ieee

Manuals? Resumes? Other technical documents? Books?

Huh? I know of precisely zero journals/conferences where Word is commonly used (in CS at least).

How about Nature?

https://www.nature.com/srep/author-instructions/submission-g...

"Whilst Microsoft Word is preferred we also accept LaTeX, or PDF format."

In general Word has eaten up everything that isn't hard core math.

Nope. Like I said all of CS uses LaTeX.

Not all of CS. It really depends on the sub field. But I would say most papers in CS are written in Latex though.

Yes, because one field uses it, the rest of the universe is irrelevant.

(PS: All the IEEE journals accept Word.

I think Word is more common in humanities and medical fields. In science and math, LaTeX is still the standard.

I work for a university press and virtually all of our manuscripts come in as Word docs, but most of our books are indeed in humanities, most are not esp. math intensive. BUt now even Word can handle math markup well enough for say, an economics text. We occasionally get LaTex files, and it is difficult to integrate into our design workflow, they have to be handled completely separately.

What? Most of my peers would laugh themselves silly if a journal asked for a word document.

I was doing a PhD in CS and one of the professors asked me to stop using LaTeX as it is “awful”.

That's a very rare attitude I think.

Over the last 10 years or so there has definitely been a push by journals more towards word.

I know that for some journals (osapublishing.org, which is optics so engineering/physics) this came at a time when they moved all their content to be accessible as html. I remember reading somewhere (maybe in their style guides) that they said that word allows them to extract the relevant information more easily, so I suspect that MS made some sort of publishing toolchain for this usecase and the publishers lapped it up.

I know that a significant portion of colleagues still prefer to write their documents in latex, and the journals still accept them.

“Most journals prefer word” well it depends of the field... In mine, computational math, it’s 100% latex. Of course it’s a math heavy field.

almost all papers on arXiv are done in latex, you upload the source and they compile it on their servers. that’s a lot of papers! it’s most of math, CS, and physics, and some economics.

it’s probably true that most papers aren’t written in LaTeX but I’m confident most people writing in LaTeX are writing scientific papers.

Even my undergraduate CS and EE classes expected me to turn in assignments in LaTeX. Which field are you working in?

Economics is mostly LaTeX.

Lua is orders of magnitude easier to learn than TeX. No macros, no expansion trickery and no obscure tricks to do things as complex calculations (either for typesetting or drawing).

Who is writing LaTeX? Mathematicians. They don't really care about the technical level of its implementation, and they surely don't care that you "don't necessarily write TeX anymore", actually, that's a negative for them.

Scientists use TeX for writing papers that they can submit to journals. As I already said, the journals don't accept ConTeXt. So of course mathematicians will rarely use ConTeXt.

> care about the technical level of its implementation

It's not about the implementation, it's about the interface.

Some journals accept Tex files? I’m surprised. SIAM (applied math) does not for instance. I don’t think IEEE does. Do Math journal accept tex files directly?

Arxiv does of course, most notably. But it’s not a journal.

> SIAM (applied math) does not for instance.

What? I have papers there and they insisted on latex.

https://www.siam.org/publications/journals/siam-journal-on-m...

> Authors of accepted papers must submit TeX files to SIAM for typesetting. Authors are highly encouraged to prepare their papers using SIAM's standard LaTeX 2e macros. Note that the SIAM office will format Plain TeX and AMSTeX files to LaTeX 2e. The LaTeX 2e macro package and documentation are available here or by email request.

That’s weird. I always upload PDF. And the copy editing is done on their side by editing the PDF using some fancy Adobe pro tool. I never uploaded a *.tex to SIAM. Of course I use latex to prepare the PDF.

> Some journals accept Tex files? I’m surprised. SIAM (applied math) does not for instance.

Each journal I've checked on their site (admittedly, only 4 or 5) seems to mandate TeX.

E.g. https://www.siam.org/publications/journals/siam-journal-on-a...

> Authors of accepted papers must submit TeX files to SIAM for typesetting. Authors are highly encouraged to prepare their papers using SIAM's multimedia LaTeX 2e macros. Note that the SIAM office will format Plain TeX and AMSTeX files to LaTeX 2e. The LaTeX 2e macro package and documentation are available here or by email request.

They definitively don’t mandate it. I got 4 papers there, all uploaded in PDF directly.

I loved using LaTeX for note taking back in university for technical classes since the mathematics typsetting is quite developed and clear - that said the fields a bit more crowded now with some nice alternatives for mathematical markup.

What are the alternatives?

AsciiMath and MathML - MathML is probably not a good dialect to use as a hand written notation but AsciiMath is pretty approachable.

TeX is still pretty darn good though.

AsciiMath is great! Thanks for the pointer.

Such awful names from a search engine perspective. Context returns the common English word, LMTX returns some Alzheimer's medication.

For a while I helped run CTAN, the TeX archive site. So my name was all over the Interwebs associated with the word "LaTeX." I got some pretty wild mail.

The usual remedy for such ills is to add more search terms. Try "typesetting", for example.

Most of "context tex" search results are relevant/useful.

This is a good point. I think products like these have strong network externalities (more the number of people use it, more beneficial they become). For this reason, a new product may not replace an incumbent product even when the incumbent product is slightly inferior. I spent a lot of time learning and using ConTeXt. Wrote one of my articles using that. When the time came to send it to a journal, they wanted it either in Word, or formatted using their LaTeX style files. At that time I gave up on ConTeXt, since I did not want to keep learning and using two different systems.

> How does that happen?

Is it easy to convert LaTeX to ConTeXt?

It's not a conversion. It's basically a separate language.

(Well, technically, the same language, but imagine all the std. library functions are different).

They're also very different philosophies.

LaTeX tries to do do everything semantically, and hide the details.

CoNTeXt otoh admits that concreteness matters, so you generally say what you want. Think "empahasis" vs "bold, underlined, 15pt text".

Yeah, I'm guessing that's probably why people aren't migrating. When there's no easy upgrade path and not a compelling enough gain, people will want to stick with the previous technology.

THe gain is very compelling, but that's sort of the point.

This isn't like going from gif to png.

It's more like going from bmp to svg.

The LaTeX (declarative) approach is better. It's so much easier to write consistent, professional looking documents that way. The other (imperative) way is more like MS Word, and we all know how that turns out. Other declarative styles that are popular include markdown and html.

So what you're saying is that LaTeX is ConTeXt free?