News for hackers. Stuff that matters.
Re:Reimplementing TeX's Algorithms: Looking Back at Thirty Y... by arnsholt on Monday January 12, 2015 @05:06PM (#8876744)
TeX is weird. Really weird, if one expects it to work like a modern programming language.

The crux of it is that it's a macro-expansion language, which means that you pretty much need to do the kind of stream-processing approach TeX uses, since you don't know the expansion of a macro up-front, which in turn can very well change the meaning of tokens downstream of that again. For an (admittedly extreme) example of this, consider TikZ; it's basically an entire new programming language, implemented purely by TeX macros.

I sometimes compare it to programming in Common Lisp, but you all you have available are the primitives (except lambda, I supppose) and defmacro.

Re:Reimplementing TeX's Algorithms: Looking Back at Thirty Y... by mturmon on Tuesday January 13, 2015 @12:27PM (#8881209)
Thanks, I think that's pretty much the issue. Because of the flexibility of the language (not just macros, but individual characters can change meaning using the \catcode mechanism), you can't have a fixed lexer. This makes the situation unlike the standard language setups I'm used to.