News for hackers. Stuff that matters.
Re:Is there a BNF grammar of the TeX language? (2010) by choeger on Sunday June 09, 2024 @10:32AM (#40624677)
It's a shame that TeX the great typesetting system comes with TeX the ungodly complex language. There's literally no chance to create good tooling for TeX because of this. There's very little chance to optimize rendering of large documents because of it (e.g., by automatic parallelization). Finally, there's never going to be a readable error message from TeX.

TeX the language is the prime example of gatekeeping.

OTOH, no one has ever succeeded in creating and popularizing a context-free language that compiles to TeX.

Re:Is there a BNF grammar of the TeX language? (2010) by omoikane on Sunday June 09, 2024 @11:49AM (#40625247)
I don't think a language is complex just because it's not context free, since any language that supports syntactic macros of some sort are not context free. Neither M4 nor C++ are context free, but C++ is usually considered more complex than M4.

Still, multiple parsers and great tooling exist for C++, and the error messages have gotten much better in recent years. This is probably because companies needed them for different reasons, and they have the resources to build alternatives.

Re:Is there a BNF grammar of the TeX language? (2010) by jjmarr on Sunday June 09, 2024 @11:04AM (#40624906)
People say TeX doesn't have good tooling but I can't think of any other language where I can seamlessly intermix circuit diagrams with equations and plots. It's really nice to be able to draw on an axis coordinate system (pgfplots with TikZ), as an example.

The language itself is horrible and the skill floor is too high but the Turing completeness enables a ton of wacky features that would never be allowed in something like Mathematica.

Re:Is there a BNF grammar of the TeX language? (2010) by g15jv2dp on Sunday June 09, 2024 @11:14AM (#40624971)
I think you're misunderstanding what "tooling" means here. Even determining what line (or even file!) in your document is the source of an error or warning is sometimes a nightmare.
Re:Is there a BNF grammar of the TeX language? (2010) by dhosek on Sunday June 09, 2024 @10:45AM (#40624788)
One of the things I decided to do with finl was to formalize on LaTeX’s syntax. No more shifting category codes, Separating out the lexer definition of commands from their meaning also means that tooling becomes simpler. Throw in proper variables so there’s no longer the need for redefining macros to emulate variables and it becomes a little more tractable.

That said, even with these restrictions, writing a good lexer/parser of a LaTeX-like syntax is still a challenge (and made all the slower by the needs to earn money, spend time with my kids and have other joys in life).

Re:Is there a BNF grammar of the TeX language? (2010) by SassyBird on Sunday June 09, 2024 @10:36AM (#40624713)
> Finally, there's never going to be a readable error message from LaTeX.

FTFY. If you learn TeX from Knuth’s TeX book, along with the typographical concepts it works with, the error messages generated by documents written in raw TeX are understandable.

LaTeX on the other hand is what the metaprogramming-heavy Boost/STL libraries are to C or C++. Abandon all hope.

Re:Is there a BNF grammar of the TeX language? (2010) by breck on Sunday June 09, 2024 @10:58AM (#40624876)
> If you learn TeX from Knuth’s TeX book

https://visualmatheditor.equatheque.net/doc/texbook.pdf

+1 on this.

That book is fantastic, one of my favorites. It's been a couple of years since I last read it, and your comment reminded me that I should probably skim again.

Re:Is there a BNF grammar of the TeX language? (2010) by WillAdams on Sunday June 09, 2024 @11:16AM (#40624987)
Except that PDF shouldn't exist since it was generated in flagrant disregard for the copyright/comment:

    % This manual is copyright (C) 1984 by the American Mathematical Society.
    % All rights are reserved!
    % The file is distributed only for people to see its examples of TeX input,
    % not for use in the preparation of books like The TeXbook.
    % Permission for any other use of this file must be obtained in writing
    % from the copyright holder and also from the publisher (Addison-Wesley).
    \loop\iftrue
      \errmessage{This manual is copyrighted and should not be TeXed}\repeat
If you want a free PDF on TeX try _TeX for the Impatient_:

https://tug.ctan.org/info/impatient/book.pdf

If you want a copy of _The TeXbook_, buy it.

Re:Is there a BNF grammar of the TeX language? (2010) by wrs on Sunday June 09, 2024 @01:40PM (#40626080)
I love the subtle humor in misusing “copyright” as a verb and then misusing “TeX” the same way.
Re:Is there a BNF grammar of the TeX language? (2010) by dunham on Sunday June 09, 2024 @01:41PM (#40626093)
Yeah, I always used plain TeX back in the day because I could understand it. LaTeX always seemed unnecessarily complex.
Re:Is there a BNF grammar of the TeX language? (2010) by breck on Sunday June 09, 2024 @10:55AM (#40624860)
> It's a shame that TeX the great typesetting system comes with TeX the ungodly complex language

You hit the nail on the head. Brilliant design of the typesetting system, not an ergonomic language.

I think over the next few years Scroll (https://scroll.pub/) will become the language all academics use to publish papers.(Or another markdown like alternative, but ours will be _very_, _very_ good)

You are always in an ergonomic, simple, fun, forgiving high level language (Scroll), but then you can dip down into TeX whenever you want for its beautiful 2D math rendering (thanks to KaTeX library).

We don't have great automatic whole document PDF generation yet (you still have to manually save as PDF from your web browser), but we will get there.

If anyone really wants this please shoot me an email--fun to get more passionate beta users that can help us steer priorities and build something free and great for paper writers all across the globe.

Re:Is there a BNF grammar of the TeX language? (2010) by me_vinayakakv on Sunday June 09, 2024 @11:21AM (#40625033)
Typst[1] is another tool which implements document generation from the ground up.

Zerodha had a great article [2] how they migrated from LaTeX based pdf generation to Typst, which ended up saving time and compute.

[1]: https://typst.app/ [2]: https://zerodha.tech/blog/1-5-million-pdfs-in-25-minutes/

Re:Is there a BNF grammar of the TeX language? (2010) by graycat on Sunday June 09, 2024 @01:44PM (#40626112)
> TeX the ungodly complex language

A friend, although he is a good mathematician, kept criticizing TeX, so I explained one view of it: TeX was not intended to be some new big steps forward in interactive user interface, to replace Microsoft's Word, HTML or Mathematica, or for making images or movies. Instead, TeX was, first, for something he knew well -- the journal of the AMS (American Mathematical Society). For doing the word processing for the AMS, TeX was a nice step forward from what went before.

TeX seems fine to me, just the way it is. Started with Knuth's book and also got his file of TeX source his book. The book is easier to read, but with my favorite text editor KEDIT sometimes the file is easier to search.

Also with the TeX software have the Windows program to display the formatted output on a screen and the program to write the output to a PDF file. Both work fine.

It appears that Knuth made commendable efforts to get any bugs out of TeX.

Wrote some macros for TeX: The main collection has, just checked, 86 macros. Also have macros for some special purposes, e.g., foils, inserting images, verbatim, business cards, etc.

Use TeX for business letters; used TeX for the last paper I published: The paper had a lot of tricky math notation, and TeX handled that well. The journal seemed quite pleased to get the TeX input with my macros; then they were able to format the paper right away.

When rusty with TeX, my old papers, letters, etc. and macro collections get me productive again quickly.

Since I like TeX so much and since the philosophy of LaTeX seems different, never used LaTeX.

Knuth's book shows that it's possible to write long documents with TeX.

Re:Is there a BNF grammar of the TeX language? (2010) by WillAdams on Monday June 10, 2024 @09:59AM (#40633769)
More importantly, TeX resulted in the concept of "Literate Programming" as exemplified by the books:

_TeX: The Program_

and

_METAFONT: The Program_

I've found that LP allows me to be successful working on projects which are more complex and of greater length than other tools allow me to --- worst case is I find myself reading through a nicely typeset hyperlinked PDF until I understand the code I need to modify and am then able to control/command click to go right to it in the editor.

I've been trying to put together a list of Literate Programs published as books:

https://www.goodreads.com/review/list/21394355-william-adams...

Re:Is there a BNF grammar of the TeX language? (2010) by graycat on Monday June 10, 2024 @11:36PM (#40642023)
Knuth's "Literate Programming"???? NO WAY!!! Not a chance!! I won't mention that one!!!! It appears that all the overt, happy, productive literate programming people get banished to, to where, to, maybe, Antarctica, under one of the ice sheets, naw, the back side of the Moon!!! E.g., for a sample of just 1, the little piece of code I'm working on now, just had my favorite text editor KEDIT count, has (hush, secret) 387 lines of comments in a file of 840 lines. And one of my worst transgressions of solid, world-wide norms, I have no undefined technical terms or acronyms. And for this code, I have an external document on the architecture -- for that transgression, could be sent to Pluto???
Re:Is there a BNF grammar of the TeX language? (2010) by WillAdams on Tuesday June 11, 2024 @08:24AM (#40645394)
I wish that I was a better programmer, so could be a better advocate for this programming style.

I wish that there were more standard tools for it --- it kills me that I couldn't find a nice .layout for doing Literate Programming in LyX w/ arbitrary programming languages.

I'm just glad that my using this system lets me overcome hurdles/complexity beyond levels which previously posed insurmountable stumbling blocks --- at a previous job there was a typesetting back-end for phone book ads which I had written in LaTeX using a documented TeX file (.dtx) --- once a year or so they would sell a license to a new customer and I would have to update it to their needs, and every quarter or so there would be a bug report... being able to read through the code would quickly get me in a place where I could fix things and then move on to the next task.

Current effort is at:

https://github.com/WillAdams/gcodepreview

At some point in time I need to look into if using Lua would allow some niceties such as per-file line # tracking...

Re:Is there a BNF grammar of the TeX language? (2010) by metaed on Tuesday June 11, 2024 @11:34AM (#40647492)
The answer to "where" is <news:comp.programming.literate> and <https://www.reddit.com/r/LitProg/>.
Re:Is there a BNF grammar of the TeX language? (2010) by cryptonector on Sunday June 09, 2024 @07:18PM (#40628421)
Some people say that TeX lacks tooling the same way that people say that of ASN.1. Those are the people that then go on to reinvent the same thing badly, like ProtocolBuffers (which is remarkably similar to DER but with different syntax).
Re:Is there a BNF grammar of the TeX language? (2010) by WillAdams on Monday June 10, 2024 @10:01AM (#40633795)
One of the things which Knuth expected was that folks would take the TeX source code and create customized versions for specific typesetting tasks.