I use exactly that. It's fine for me and I'm very much a terminal first person.

Maybe people have extremely fancy shell setups? Perhaps it's a font thing that slows it down?

I just timed my xterm. I'm displaying about 2.3MB/s of characters. I really don't know what people are doing where this is slow - I don't think even Evelyn Wood claimed to read at that speed (also it's worth noting I'm tech-old and grew up on 2400/9600 baud so this will forever calibrate my expectations at some deep subliminal level)

Basically here was my test

    $ dd if=/dev/urandom bs=10M count=1 | hexdump -C > /tmp/test
    $ ls -l /tmp/test
    -rw-r--r-- 1 chris chris 51773449 Oct  8 09:32 /tmp/test
    $ time cat /tmp/test

    ... wait wait wait ....

    51773449 / (total time) / (2^20)
   

Now this of course depends on the size of the window. I'm on a 4320x3840 dual 4k monitor and the window geometry was 1334x928 for this test.

I acknowledge this computer (i7-8650U ~ 2018 thinkpad) is capable of faster output but the xterm featureset is vaaassstt (albeit extremely complicated) and I've found the other terminals to be lacking.

2MB/s is glacial. Your computer is very fast, it should be able to print over 1GB/s without breaking a sweat.

Casey Muratori demonstrated this with his refterm project. It seems like a lot of interest in making fast terminals has followed from that.

Why do I care? I can't read things that fast

This isn't rhetorical. I can't answer the question

To be honest, refterm was a bad idea to begin with. It's not a terminal, and it eventually also stated that in its README.md. Not to mention, that you - by definition - can't compare apples with pies. refterm is not a terminal and thus cannot be compared with a terminal WRT speed. It simply prove a point on rendering speed, at a cost of breaking a lot of the rules a terminal emulator has to obey to. Most notably is `DECAWM` (DEC Auto-Wrap Mode) that was not properly implemented. The author once stated in a Twitch stream that he simply didn't understand that and thus skipped that part. However, that is the part where it will get complex (and thus slow) if you want to get it right for non-trivial US-ASCII characters, such as complex grapheme clusters (family Emoji most interestingly) or even VS16 (variation selector changing the width of the base code point from narrow to wide). The latter point isn't standardized by DEC anyways, but the younger terminals all try to make the terminal more appealing to the younger Unicode crowd that do expect complex Unicode to render "just fine" (or "as fine as it gets"). refterm also had a hard coded 4k line length limit and just hard-cut off there instead of properly counting for the wrap.

TL;DR refterm (while positively inspirational to a few) was harmful to the terminal community, because it was fooling a lot of people with false metrics that the users simply believed in, without questioning.

I use xterm. Seems fine. Not sure what I'm missing I guess but I get my work done.