Re:In the Turkish locale, "INFO".lower() != "info"
by kentonv on Sunday August 16, 2020 @01:55PM
(#24179727)
The code is not broken for non-ASCII characters. UTF-8 works just fine with 8-bit chars, and the code I wrote correctly lower-cases ASCII letters even when UTF-8 is present (it just won't touch the UTF-8 chars, which is fine in this use case).
It's only tolower() and toupper() specifically that are broken because they expect to be able to do their job on a single byte, which is no longer possible with UTF-8.
Meanwhile, using tolower() to lower-case an HTTP header name won't give you the correct results if the locale is set to Turkish with the ISO 8859-9 character set, which is 8-bit, and where tolower('I') will produce the byte 0xFD which is 'ı' in this character set.
It's only tolower() and toupper() specifically that are broken because they expect to be able to do their job on a single byte, which is no longer possible with UTF-8.
Meanwhile, using tolower() to lower-case an HTTP header name won't give you the correct results if the locale is set to Turkish with the ISO 8859-9 character set, which is 8-bit, and where tolower('I') will produce the byte 0xFD which is 'ı' in this character set.