Re:A popular but wrong way to convert a string to uppercase ...
by squeaky-clean on Wednesday October 09, 2024 @08:40AM
(#41787266)
Yeah If you're using narrow strings for UTF8 you're making a mistake. wstrings also are not a good representation because of the platform differences, unless you don't care about Windows in which case it's fine but still not great semantically.
In most type definitions you cannot convert UTF8 via simple iteration because the type generally represents a code point and not a character.
You can have a library where UTF8 characters are a native type and code points are a mostly-hidden internal element. But again, that's highly opinionated for C++.
Re:A popular but wrong way to convert a string to uppercase ...
by gpderetta on Thursday October 10, 2024 @05:27AM
(#41797113)
I'm not 100% sure what you mean by narrow string, but if you refer to std::string vs std::wstring, then std::string is perfectly fine for encoding UTF8, as that uses 8 bit code units which are guaranteed to fit in a char. On the other hand, std::wstring would be a bizarre choice for UTF8 on any platform.
Re:A popular but wrong way to convert a string to uppercase ...
by account42 on Wednesday October 09, 2024 @08:44AM
(#41787314)
It's not guaranteed for 7-bit ASCII either because tolower/toupper are locale-dependent and with the tr_TR lowercase I (U+0049) is ı (U+0131, aka dotless i) wich encodes as two bytes in UTF-8.
Re:A popular but wrong way to convert a string to uppercase ...
by squeaky-clean on Wednesday October 09, 2024 @09:02AM
(#41787482)
That's not ascii then. It's byte width compatible (to a certain degree as you point out). But it's not ascii. ascii defines 128 code points and the handling of an escape character. It doesn't handle locales.
Since you mention narrow strings in the context of wstring, just to make sure... you can't convert a UTF-8 std::string character by character, in-place (in case that's what you meant).
7-bit ASCII code points are fine, but outside that it's not guaranteed that one UTF-8 byte converts into exactly one UTF-8 byte when converting case.