I'm quite sure anybody who has written substantial documents with *TeX disagrees that "extreme performance" isn't needed.

The average document size hasn't increased in the last 30 years, though, did it? But hardware is several orders of magnitude faster.

I guess we're doing more with graphics, which tend to require more fiddling back and forth to make it look alright, as well as having much higher expectations about interactivity than in the past.

In TeX? My impression was that it's still mostly used for papers, and the output is still normally static DVI/PS/PDF.

LaTeX is a typesetting ststem, not a documentation system. Doing typesetting of large documents is performance critical.

If typesetting is performance critical, what would count as a non-performance critical program?