News for hackers. Stuff that matters.
Re:The lack of a case against Python 3 by lorenzfx on Tuesday November 29, 2016 @04:25AM (#13061895)
> But again, the population that will be affected is the 10% who deal with Unicode.

I doubt that only 10% of people are dealing with unicode. Rough estimate 10% of the world speak English as a first language and almost all other languages are written with (at least some) non-ascii characters.

In my experience (I'm from the 90% part of the world), people new to programming and python stumble really early into some (to them) hard to debug (UnicodeDecodeError: ... can't decode byte) errors in python 2. Then it's time to explain the whole strings are not bytes, unicode, encoding and so on thing. In python 3 that's also needed, perhaps even earlier, but bugs are much easier to fix and programs fail on the first run, not when you finally enter some non-ascii characters.

In my mind, python 3 clearly is superior to python 2, and would be even if nothing but the unicode support changed.

Re:The lack of a case against Python 3 by gardarh on Tuesday November 29, 2016 @05:59AM (#13062281)
Actually, I think the strictness of python3 with regards to bytes/strings is particularly useful for people who speak English as their first language. python3 will force them into writing software that can work with international strings whereas python2 code written by an English speaking person would in many cases just be broken for the international users.

And believe me, as an "international" person, this is profoundly annoying and has trained me in avoiding using the non-ASCII letters of my language which of course "castrates" the written language somewhat (i.e. When people use non-ASCII letters for folder names I think they are just asking for trouble - how normal is that?)

I love how upfront python3 is with the difference between raw bytes and strings. I work a lot with python and in particular I do a lot of work with serializing and deserializing stuff from/to binary blobs. Due to platform issues that I hope will change in the next year or so I can't use python3 currently but I keep all my code compatible with both and all tests should run with python2 and python3. Yes, I am firmly in the python3 camp :)

pycharm inspection definitely helps with keeping code compatible with 2 and 3.

Re:The lack of a case against Python 3 by cabalamat on Tuesday November 29, 2016 @09:14AM (#13063116)
> When people use non-ASCII letters for folder names I think they are just asking for trouble

Well they are

> how normal is that?

Not as normal as it ought to be.

Folder names (and file names) are identifiers. Identifiers should identify. If you allow unicode characters, then you can have two character sequences that look the same but are actually distinct. This is confusing at best and at worst (in URLs) could facilitate fraud.

Re:The lack of a case against Python 3 by Xylakant on Tuesday November 29, 2016 @09:33AM (#13063222)
> If you allow unicode characters, then you can have two character sequences that look the same but are actually distinct.

That would be a software bug. If you want to compare unicode strings, you need to normalize them first following the rules laid out in the standard. https://en.wikipedia.org/wiki/Unicode_equivalence

git for example fails that test. (try creating a repo with a file named 'ü' and check it out both on a mac and a linux system)

There are more issues about glyphs that are a distinct character but look the same in a given font, but what's your proposal? All people transliterate everything to ascii? Display punycode in URLs?

Re:The lack of a case against Python 3 by pjc50 on Tuesday November 29, 2016 @09:47AM (#13063295)
> Identifiers should identify

Except that, without Unicode, you now have to use a slighly restricted subset of many European languages and cannot use Cyrillic, Chinese etc at all. So people who are not operating in an English environment are limited in their ability to clearly identify things.

Re:The lack of a case against Python 3 by Someone on Tuesday November 29, 2016 @07:14PM (#13067381)
"then you can have two character sequences that look the same but are actually distinct."

If you accept that argument, you should also remove either 1 or l, O or 0, S or 5 etc.

Re:The lack of a case against Python 3 by JimDabell on Tuesday November 29, 2016 @04:58AM (#13062041)
> Rough estimate 10% of the world speak English as a first language

ASCII isn't even good enough for that 10% – British people need £, which isn't in ASCII – not to mention things like em dashes, en dashes, and curly quotation marks that word processors tend to add.

Re:The lack of a case against Python 3 by JulianMorrison on Tuesday November 29, 2016 @05:33AM (#13062183)
They're in latin-1 though, which is a byte-per-character encoding.
Re:The lack of a case against Python 3 by herge on Tuesday November 29, 2016 @08:20AM (#13062843)
I don't think anyone who deals with character encodings should encourage latin-1 in 2016 instead of utf8. Fair enough, your old C code will work, but everyone else will hate you.

Except, of course, if you deal with Windows...

Re:The lack of a case against Python 3 by pjc50 on Tuesday November 29, 2016 @09:48AM (#13063306)
.. in which case it's Windows-1252 rather than latin-1. And maybe you have to deal with UCS-2, god help you.
Re:The lack of a case against Python 3 by friendzis on Wednesday November 30, 2016 @03:40AM (#13069301)
Except that encoding depends on your regional settings, which in Baltics is Windows-1257 ;)
Re:The lack of a case against Python 3 by guan on Tuesday November 29, 2016 @07:59AM (#13062740)
Kids these days also use emoji.
Re:The lack of a case against Python 3 by yummyfajitas on Tuesday November 29, 2016 @08:20AM (#13062835)
Re:The lack of a case against Python 3 by rmc on Thursday December 08, 2016 @07:53AM (#13129971)
Emoji have been part of Unicode since 1993 (e.g. ️)
Re:The lack of a case against Python 3 by CalRobert on Tuesday November 29, 2016 @04:38AM (#13061943)
I loved python2, but now that I'm in a company where everything is python3 I probably save a few days a year of not mucking about with Unicode BS. It's wonderful. There may be other advantages but even if it were JUST "python2 without horrible Unicode idiosyncrasies" it would be worth it.
Re:The lack of a case against Python 3 by hobarrera on Tuesday November 29, 2016 @07:57AM (#13062734)
> I doubt that only 10% of people are dealing with unicode. Rough estimate 10% of the world speak English as a first language and almost all other languages are written with (at least some) non-ascii characters.

I believe he meant people that have to actually deal with complications of unicode/bytestring themselves.

For most uses, you just handle "strings", and don't really care if they're unicode or not. Only lower-level libraries/apps will ever care if these strings are unicode or not.

Re:The lack of a case against Python 3 by reuven on Tuesday November 29, 2016 @08:34AM (#13062908)
Yup, that's what I meant!
Re:The lack of a case against Python 3 by zamalek on Tuesday November 29, 2016 @09:16AM (#13063132)
> I doubt that only 10% of people are dealing with unicode.

I will use a single character to express my absolute agreement:

Edit: I guess even Lisp developers have to consider Unicode.

Re:The lack of a case against Python 3 by lispm on Tuesday November 29, 2016 @04:04PM (#13066275)
True.

Even if your Lisp supports unicode, it does not make a web framework/application support unicode just so.

Disclaimer: I've added UTF-8 support to a Lisp-based web framework, on top of a Lisp implementation supporting Unicode.

Re:The lack of a case against Python 3 by junke on Tuesday November 29, 2016 @09:56AM (#13063354)
> Edit: I guess even Lisp developers have to consider Unicode.

Why wouldn't they?

Re:The lack of a case against Python 3 by WorldMaker on Tuesday November 29, 2016 @01:30PM (#13065060)
To explain/spoil the joke: HN does not support most of Unicode and is written in a Lisp variant.
Re:The lack of a case against Python 3 by junke on Tuesday November 29, 2016 @01:57PM (#13065317)
Thanks, I clearly didn't get it :-)
Re:The lack of a case against Python 3 by IndianAstronaut on Tuesday November 29, 2016 @10:51AM (#13063761)
Does anyone know what underlying differences there are in Python 3 unicode handling versus Go's rune system. I have found the latter far easier to work with when dealing with text characters in other languages.