News for hackers. Stuff that matters.
Re:UTF-8 Everywhere (2012) by nailer on Sunday June 19, 2016 @05:20PM (#11934728)
I haven't done Python for a while, so maybe it's changed, but I thought the new Python 3 strings werre index by byte rather than character, and it was 'as designed' despite being very non-intuitive (unlike the rest of Python).

Edit: see pjscott's comment below - it's by code point, not byte, but still not by character.

Re:UTF-8 Everywhere (2012) by Animats on Sunday June 19, 2016 @06:03PM (#11934919)
No, Python 3 indexes by character, not byte.

    $ python3
    Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
    >>> s = "オンライン"
    >>> s
    'オンライン'
    >>> len(s)
    5
    >>> s [0]
    'オ'
    >>> s [1]
    'ン'
    >>> s [2]
    'ラ'
    >>> s[3]
    'イ'
    >>> s[4]
    'ン'
Re:UTF-8 Everywhere (2012) by pjscott on Sunday June 19, 2016 @06:14PM (#11934963)
It's important to be insufferably pedantic about this: they index by code point, which is almost but not quite what people expect a character to be.

    $ python3
    >>> "위키백과"[1]
    '키'
    >>> "위키백과"[1] # Should be identical, right?
    'ᅱ'
Re:UTF-8 Everywhere (2012) by kbenson on Sunday June 19, 2016 @07:56PM (#11935351)
> It's important to be insufferably pedantic about this

That is perhaps the most succinct and accurate way I've heard to explain and justify why you're sounding like a wet blanket to people that may not understand, while acknowledging that you know how you sound, but there is a reason for it. I expect to use this in the future.

Re:UTF-8 Everywhere (2012) by schoen on Sunday June 19, 2016 @11:40PM (#11935919)
That's an awesome example! To show more of how it works for Western language speakers who might be confused, how about

c = "é"

c[0], c[1]

It's the same phenomenon with Latin characters. (Extra bonus: for me, the combining acute accent character then combines in the terminal with the apostrophe that Python uses to delimit the string!)

Another idea to see the effect is "a" + "é"[1]. (The result is 'á'... and as in your examples, a precomposed "̈́é" is also available which doesn't exhibit any of these phenomena.)

Re:UTF-8 Everywhere (2012) by kazinator on Sunday June 19, 2016 @10:26PM (#11935750)
Same with English, sort of. What is

    "difficult"[2]?
Is it 'f'? Or the ligature `ffi`? :)