They probably went to efforts to make it work an ancient system. But you really didn't have to.
Tunneling through the layer of crust accumulated in order to support Unicode (several layers of crust as Unicode evolved) was probably the most interesting of all.
A competent programmer could fit the whole charset in their head and easily write a TUI from scratch in a day. But they didn’t need to: plenty of great libraries existed.
Everything is so much easier when you have standardized hardware instead of layers upon layers of abstractions supporting all sorts of weird things, like we have to put up with this century.
ncurses renders to an underlying framebuffer which contains 32-bit Unicode characters, plus attributes (FG/BG color pair, and 8 additional attributes like underline, bold, blink). I'm not sure which layer of crust this framebuffer belongs to -- something related to TERMINFO, I think. The intermediate framebuffer is then rendered to the actual VGA framebuffer in a separate pass, which maps 32-bit unicode characters onto glyph indexes in the currently-loaded VGA font.
There are some provisions for Asian character sets (Vietnamese, for example, is plausibly supported). I'm not sure if Chinese or Japanese VGA terminals were ever adequately supported in Linux.
Can you elaborate on that? Are your "clumsy scripts" written in Python, shell or another language? Have you tried Textual-based tools like Trogon [1] and Moulti? [2]
[1] https://github.com/Textualize/trogon [2] https://moulti.run/
Thanks a lot for sharing these, especially Trogon, which I think is a clever idea with high ROI.
Then I found Textual (Python library, https://textual.textualize.io/) which feels like its modern and prettier child. Not gonna lie, it's still not a turnkey solution that transformed my clumsy scripts into a beautiful TUI without work but it's close!