Re:Zig 0.9.0
by fastball on Monday December 20, 2021 @09:25PM
(#29633100)
I don't think I've ever had a problem with Python3's string handling, which is very robust.
[ Parent
| on HN ]
>>> "ñ"[0]
'n'
>>> "ñ"[1]
'̃'Huh, which version is that? Python 3.9 on my system:
>>> "ñ"[0]
'ñ'
>>> "ñ"[1]
IndexError: string index out of range
Which is what I would expect.this is normalization issue, not version issue
import unicodedata list(unicodedata.normalize('NFD', "ñ")) >> ['n', '̃']
list(unicodedata.normalize('NFC', "ñ")) >> ['ñ']
both are correct, the issue is that unicode allows accented letters to be written as _accented_letter_ or _letter_, _accent_. The idea of "character" in uncicode is not very useful, most of the time you will want graphemes, not codepoints. User-friendliness wise, this is what Python should use (another rant - strings should not have length method, they should have byte_length, codepoint_length and grapheme_length).
Python 3.9.9 (main, Nov 20 2021, 21:30:06)
[GCC 11.1.0] on linux
EDIT: I can't seem to replicate this on my Ubuntu system. Strange.
Ah, interesting, I suppose then there is a difference in the way Linux handles strings? Didn't realize that, very unfortunate if true. I am running MacOS.
Actually browser normalized string and it works for me also. Here is original byte sequence:
>>> b'n\xcc\x83'.decode()
'ñ'
>>> b'n\xcc\x83'.decode()[0]
'n'
>>> b'n\xcc\x83'.decode()[1]
'̃'
But I agree, it's rare case when you need to deal with non-normalized data.