Re:A popular but wrong way to convert a string to uppercase ...
by steveklabnik on Tuesday October 08, 2024 @08:19PM
(#41783225)
In Rust, the APIs are clear if they're ASCII only or unicode aware.
https://doc.rust-lang.org/stable/std/primitive.str.html#meth...
> ‘Lowercase’ is defined according to the terms of the Unicode Derived Core Property Lowercase.
https://doc.rust-lang.org/stable/std/primitive.str.html#meth...
> ASCII letters ‘A’ to ‘Z’ are mapped to ‘a’ to ‘z’, but non-ASCII letters are unchanged.
Now, "perfectly" is very strong. For example, the Turkish i problem. That is not solved. But 99% of Unicode stuff is handled correctly by default.
Re:A popular but wrong way to convert a string to uppercase ...
by oguz-ismail on Tuesday October 08, 2024 @10:24PM
(#41783870)
> 99% of Unicode stuff
Does that include context-dependent conversion rules like o'reilly -> O'Reilly?
Re:A popular but wrong way to convert a string to uppercase ...
by PaulDavisThe1st on Wednesday October 09, 2024 @01:10AM
(#41784679)
that is neither up-casing nor-downcasing, but (de)capitalization, which is a significantly more complex task (which ultimately requires up- or down-casing, but a whole lot more before then).
Re:A popular but wrong way to convert a string to uppercase ...
by oguz-ismail on Wednesday October 09, 2024 @01:26AM
(#41784756)
So it doesn't. If Unicode doesn't cover non-trivial forms of case-folding, 99% of Unicode doesn't mean anything.
Re:A popular but wrong way to convert a string to uppercase ...
by PaulDavisThe1st on Wednesday October 09, 2024 @01:49PM
(#41790691)
I am not aware of a Unicode concept of "the latin letter o followed by an apostrophe followed by another latin letter". Unicode would identify the glyphs for such a concept, but I don't see how Unicode is involved in this in anyway as far the process of deciding what "capitalized o'reilly" means.
Re:A popular but wrong way to convert a string to uppercase ...
by steveklabnik on Wednesday October 09, 2024 @08:04AM
(#41786951)
Sort of, see the Greek example elsewhere in this thread. I don’t think that specific situation is part of Unicode though.
[1] Ended up looking at https://github.com/JakubSzark/zig-string