Re:Unicode Normalization Forms: When ö ≠ ö
by tsimionescu on Saturday January 01, 2022 @06:12AM
(#29757395)
> and the custom to compare strings as byte arrays
I think more generally, the idea that a langauge std lib can provide string equality for human langauge strings is just silly. String equality is an extremely context dependent, fuzzy operation, and should be handled by each context differently. For example for Unicode hostname to certificate mapping, hostname equality should be handled by rendering the hostname in several common web fonts and checking if the resulting bitmaps are similar. If they are, then assigning different certificates to these equal hostanmes should not be done.
Of course, in other contexts, there are different rules. For example, if looking up song names, the strings "jalapeno" and "jalapeño" should be considered equal, in English text at least.
In any case, there are emoji which are expected to be a part of text.
On one hand, it would be great to separate areas of different encodings inside a string. But character codes are already such separators.
Two things need to go though: the assumption of linear-time index-based access to characters in a string, and the custom to compare strings as byte arrays.
The first us already gone from several advanced string implementations. The second is harder: e.g. Linux filesystems support Unicode by being encoding-agnostic and handling names as byte sequences. Reworking that would be hard if practical at all.