News for hackers. Stuff that matters.
Re:UTF-8 is a brilliant design by asddubs on Saturday September 13, 2025 @05:20AM (#45230639)
the fact that there is seemingly no interest in fixing this, and if you want chinese and japanese in the same document, you're just fucked, forever, is crazy to me.

They should add separate code points for each variant and at least make it possible to avoid the problem in new documents. I've heard the arguments against this before, but the longer you wait, the worse the problem gets.

Re:UTF-8 is a brilliant design by Cloudef on Saturday September 13, 2025 @12:05PM (#45233174)
Afaik theres some language hints nowadays but its kinda hack
Re:UTF-8 is a brilliant design by meindnoch on Saturday September 13, 2025 @01:19PM (#45233710)
What happens if you want both single-storey "a" and double-storey "a" in the same document? You use a different font.
Re:UTF-8 is a brilliant design by asddubs on Saturday September 13, 2025 @04:17PM (#45235001)
I won't even touch the fact that what you're talking about is just a stylistic difference, rather than a language based one, and will instead say this: What if you want the cyrillic letter А and the latin letter A, which are not just the same glyph, but literally visually identical looking in the same document? Oh wait both of those have separate UTF-8 codepoints. But if you want chinese and japanese characters which do not look identical in the same document, you have to resort to changing fonts? What if you're using an encoding that doesn't support specifying fonts? Your non-response doesn't solve anything and helps no one
Re:UTF-8 is a brilliant design by eviks on Saturday September 13, 2025 @01:32PM (#45233801)
Some fonts allow for both alternatives in them
Re:UTF-8 is a brilliant design by eviks on Saturday September 13, 2025 @01:34PM (#45233810)
Why is the language tag not used to signal a variant?
Re:UTF-8 is a brilliant design by jabedude on Sunday September 14, 2025 @10:40AM (#45240090)
That doesn't help in a mixed Chinese-Japanese document
Re:UTF-8 is a brilliant design by eviks on Sunday September 14, 2025 @10:42AM (#45240109)
Why not? You don't have a single tag limit per document and can tag every mixed part with the appropriate language
Re:UTF-8 is a brilliant design by jabedude on Monday September 15, 2025 @11:30AM (#45250879)
That's not the only granularity of mixed text. A Chinese textbook about the Japanese language will have sentences where the languages are mixed
Re:UTF-8 is a brilliant design by eviks on Monday September 15, 2025 @11:35AM (#45250957)
You still haven't explained what the issue is

Chinese textbook: <ch>Chinese <jp>Mixed Japanese</jp> continue Chinese.</ch>