Re:In the Turkish locale, "INFO".lower() != "info"
by ramshorns on Sunday August 16, 2020 @11:57AM
(#24178806)
A char in C++ is one byte, right? Is it even possible for this "fixed" code to call ctype::tolower() on something like a UTF-8 or UTF-16 code point?
[ Parent
| on HN ]
Re:In the Turkish locale, "INFO".lower() != "info"
by kentonv on Sunday August 16, 2020 @12:10PM
(#24178921)
Correct, it won't even work as intended with modern Unicode locales.
Re:In the Turkish locale, "INFO".lower() != "info"
by ramshorns on Sunday August 16, 2020 @12:36PM
(#24179144)
So maybe if the code is broken anyway for non-ASCII characters, it's fine to use tolower, since somewhere else in the code it ensures that c is a byte.
Re:In the Turkish locale, "INFO".lower() != "info"
by kentonv on Sunday August 16, 2020 @01:55PM
(#24179727)
The code is not broken for non-ASCII characters. UTF-8 works just fine with 8-bit chars, and the code I wrote correctly lower-cases ASCII letters even when UTF-8 is present (it just won't touch the UTF-8 chars, which is fine in this use case).
It's only tolower() and toupper() specifically that are broken because they expect to be able to do their job on a single byte, which is no longer possible with UTF-8.
Meanwhile, using tolower() to lower-case an HTTP header name won't give you the correct results if the locale is set to Turkish with the ISO 8859-9 character set, which is 8-bit, and where tolower('I') will produce the byte 0xFD which is 'ı' in this character set.