News for hackers. Stuff that matters.
Re:Python 3 makes it easier to develop high-quality software by pornel on Wednesday November 18, 2015 @11:02AM (#10588420)
This is sad and frustrating. The article is trying to convince people to upgrade to a 7-year-old version of Python! And everyone is still skeptical about it, thinking it's too new, too risky, unproven, and libraries aren't ready yet.

Python 3 adoption takes longer than entire NodeJS lifetime. In the same timeframe PHP introduced and EOLed 5.3 (edit: and there were 4 years between release of 5.0 and EOL of 4.x, which involved backwards-incompatible refactoring of OO semantics).

Re:Python 3 makes it easier to develop high-quality software by rdtsc on Wednesday November 18, 2015 @03:42PM (#10590599)
> And everyone is still skeptical about it,

They are not skeptical. Nobody thinks "yeah if I just wait long enough, Python 3 will go away and 2.8 will come out".

They just don't see an incentive to update. Simple as that. It is basic economics of time and money.

Python 2 wasn't terrible and Python 3 isn't dramatically better. Unicode and other stuff in Python 3 is nice. Is it nice enough to start digging into stable, working, money making code just to say "Yay, Python 3"? I think the development community has spoken by how it was handled. It's gonna happen but it will happen at a very slow pace.

Imagine this alternative scenario -- "Python 3 brings 3x speed improvement + removed GIL". Ok, I bet you, there would have been a much faster adoption because now there is a greater incentive to justify it. People would think "Yeah, I'll destabilize my code a bit but I gain performance, I can do that".

Someone in a sibling comment suggested "They should have just stopped updating security patches for 2.7, that would have forced them to use 3.0". That's ok if Python holds a monopoly on development languages. But it doesn't. As soon as that happens developers will not just look at updating to Python 3, they'll start looking at other ecosystems: Go, Elixir, Java, Clojure, Scala, NodeJS etc.

Re:Python 3 makes it easier to develop high-quality software by mangeletti on Wednesday November 18, 2015 @03:51PM (#10590648)
I could not agree more.

To add to this; if it included even a 2x speed up / memory reduction, I think 90% of everyone would have switched by now, and you know what else? Go probably wouldn't exist.

Re:Python 3 makes it easier to develop high-quality software by thrownaway2424 on Wednesday November 18, 2015 @05:40PM (#10591288)
This is a good point. Go is a direct response to Google's experience with Python.
Re:Python 3 makes it easier to develop high-quality software by k_sze on Wednesday November 18, 2015 @07:22PM (#10591805)
I would argue that removing the GIL would scare the shit out of developers and slow adoption instead of hasten it. A lot of accidentally thread-safe code would suddenly become thread-unsafe.
Re:Python 3 makes it easier to develop high-quality software by rdtsc on Wednesday November 18, 2015 @10:47PM (#10592608)
It was just a hypothetical example of the kinds of changes that would have flocked everyone to Python 3.
Re:Python 3 makes it easier to develop high-quality software by jrumbut on Wednesday November 18, 2015 @04:48PM (#10591012)
I think the energy spent supporting Python 2 and the inability to really unify and focus the community on Python 3 is preventing or delaying that kind of dramatic improvement.

If people would rather move to another ecosystem rather than update their code that's fine. The Python community doesn't get paid a nickel every time the interpreter runs, (at least slightly) engaged users are what move Python forward.

Re:Python 3 makes it easier to develop high-quality software by rdtsc on Wednesday November 18, 2015 @10:52PM (#10592626)
The difference is if we look at it from the point of view of the Python community (and they are all about Python 3). But another view is that of developers who see themselves as developers of product X or company Y and they value their bread on their table or that X and Y wins more than Python is popular or Python succeeds.
Re:Python 3 makes it easier to develop high-quality software by s73v3r on Wednesday November 18, 2015 @03:52PM (#10590654)
I would say that actual, continued support is an incentive to update.
Re:Python 3 makes it easier to develop high-quality software by njharman on Wednesday November 18, 2015 @12:55PM (#10589431)
I think there is way less "anti" python 3 attitude than there appears. A few haters, a few blog spam/click baiters, a few people who never even looked at python3. Vast majority of people using Python3 (me included), just use it. It's not a big deal. No reason to toot my horn over it.

There is lots of FUD from early 3.0-3 days. Python 3.5 is solid and well supported by 3rd party libs check this out

https://caniusepython3.com/

Python3 has been adopted, now. It just isn't newsworthy so we don't hear about it.

Re:Python 3 makes it easier to develop high-quality software by fredrik-j on Wednesday November 18, 2015 @02:41PM (#10590183)
While CPython 3.x is rock solid and shipped years ago - no argument there - both Pypy and Jython still lacks stable Python 3.x releases. That's a major migration pain point.

It is hard trying to introduce Python 3 at a place that has systems that depend on both Cpython and Jython. Jumping between python 2 and 3 between projects is quite taxing mentally. And staying on python 2 only leaves me with a sense of lingering doom to say nothing of having to endure all the idiosyncrasies of python 2.x. I really enjoy the few moments when I can use Python 3.

It is perhaps easy to consider the 2 vs 3 debate over and done with for people who can just disregard Pypy and Jython entirely in their environment. I probably would think so myself had I not been so unfortunate to be the unwilling maintainer of solutions dependent on Jython right now.

Re:Python 3 makes it easier to develop high-quality software by sigzero on Wednesday November 18, 2015 @03:30PM (#10590526)
Jython has always been behind and sometimes by a large margin. I don't think that is a "major" migration point at all.
Re:Python 3 makes it easier to develop high-quality software by Animats on Wednesday November 18, 2015 @01:52PM (#10589864)
No, it hasn't, based on download stats of packages. Guido found this out the hard way when he went to work for Dropbox. They run mostly Python 2.7 code, and now he has to maintain it.

Also, PyPy is 2.7 compatible, although there's a 3.x beta.

Re:Python 3 makes it easier to develop high-quality software by nostrademons on Wednesday November 18, 2015 @02:30PM (#10590108)
Guido knew this long before going to DropBox. Google is still on 2.7. IIRC I saw some mailing list posts back around 2008 that acknowledged that most large organizations will never upgrade to 3.0, because it just doesn't make sense when you have a large legacy codebase.

That doesn't make Python 3 a failure or unadopted, it's just the structure of our industry. By search & job stats [1], the most popular programming languages are Java, C, C++, C#, and yep, Python. Companies tend to pick whatever language is best for early adopters when they first get started and then language usage grows along with the company. So the most popular languages now are the ones that were cutting edge 15 years ago, when the biggest companies of today were tiny startups. Go look at what's most popular among startups today and you'll see what'll be the hot programming languages in 15 years.

[1] http://www.tiobe.com/index.php/content/paperinfo/tpci/index....

Re:Python 3 makes it easier to develop high-quality software by digaozao on Wednesday November 18, 2015 @03:18PM (#10590429)
Nice metric to see if a language is going to be mainstream. I think this should work. Do you think of any way to validate your theory?
Re:Python 3 makes it easier to develop high-quality software by nostrademons on Wednesday November 18, 2015 @03:45PM (#10590612)
You can backtest it against languages that appear frequently in programming forums or Silicon Valley meetups and then take a look at their popularity & use by big companies 5 years later.

I've done this anecdotally (i.e. not with a huge quantitative sample size, but looking at specific "hot" tech companies and the tech stacks they end up using) and it seems to work, but is subject to certain gotchas. Data points:

EBay (1995) - Perl, rewritten in C++ in 1997, rewritten in Java in 2002. Google (1998) - C++. Del.icio.us (2003) - Perl. GMail (2002) - Perl, rewritten in Java circa 2002. LiveJournal (2002) - Perl. Flickr (2004) - PHP. Facebook (2004) - PHP. YouTube (2005) - Python. Reddit (2005) - Lisp, rewritten in Python in 2006. AirBnB (2006) - Ruby on Rails. GitHub (2007) - Ruby on Rails. Dropbox (2007) - Python. Hacker News (2007) - Arc, YC internal tools (c. 2012) are written in Ruby on Rails. Twitter (2007) - Ruby on Rails, rewritten 2010 in Java/Scala. Uber (2010) - Node.js, supposedly this was a rewrite.

There are two big caveats:

Sometimes a big competitor comes along right when a language is undergoing a massive rewrite - for example, it looked like Perl would own the Internet around 2003, but just a couple years later we got Rails and Django and Perl stagnated with Perl 6, and so Perl jobs are nowhere near as hot as they would otherwise be. Similarly, Python was poised to take over much of the non-web server space in 2008, but in 2009 Node.js came out just as Python was losing steam in the Python 3 transition. Python still remains pretty hot (it's been helped by its use in data science), but it may've lost the startup server market to Node.

The other big anomaly is that there was a lot of attention around Haskell and Erlang from 2005 - 2008, and yet these are still niche languages. Erlang had one big success with Whatsapp and some smaller ones with Facebook chat and CouchDB, but didn't really become mainstream. My hypothesis there is that because both of these were old languages that were rediscovered, they were built with some assumptions (eg. strings are lists, or funky POSIX interfaces) that didn't fit modern startup development, and so they couldn't gain critical mass. But then, both Python and Ruby were old languages that were rediscovered; perhaps it was the sudden resurgence of Linux (which both Python and Ruby were steeped in) along with the web that carried them forwards.

Re:Python 3 makes it easier to develop high-quality software by rlander on Wednesday November 18, 2015 @11:53PM (#10592828)
What a great analysis!
Re:Python 3 makes it easier to develop high-quality software by mangeletti on Wednesday November 18, 2015 @04:01PM (#10590725)
So it hasn't been adopted, based on a sample size of 1?

New companies, like the one I'm part of, use Python 3, all the time. Django, and the other 15 packages we require, all support Python 3. In fact, I've been using Python 3 since 2008, and I've been using it for work since 2013. I think we're past all of that, at this point.

I'm not sure why migration is always regarded as a metric for succes. Python 3 wasn't supposed to be a "upgrade", per se. It was supposed to allow the core to rid itself of a lot of the language's warts. It has done this quite well without sacrificing much. The result was a slow migration, not by existing code bases, but by means of new companies.

Re:Python 3 makes it easier to develop high-quality software by macavity23 on Wednesday November 18, 2015 @02:16PM (#10590006)
PyPy3 stable is compatible with Python 3. PyPy4 is not, yet.
Re:Python 3 makes it easier to develop high-quality software by vegabook on Wednesday November 18, 2015 @01:07PM (#10589526)
This is true and incomplete at the same time.

It's true that 3 is happily happening for many people, effectively and without fuss.

But it's incomplete to pretend the schism is not an issue; not to try to understand the pro-2.x points which many reasonable users have.

Its a huge issue, even now, and to pretend it's not there is to do BOTH pythons a disservice because it will simply serve to feed the disagreement, to the detriment of both flavours of the language.

The Python 3 project needs to understand where it went wrong, acknowledge it, 'better language' arguments (true-ish but largely ineffective) notwithstanding, and try to bridge the community.

Yes the 3 segment reasonably wishes all this would just go away so we can all focus on moving python forward, but it's not going away. It must be adressed without yet another post about 3 being better, which simply makes 2.7 people feel like second class citizens. They've heard all the arguments before.

Re:Python 3 makes it easier to develop high-quality software by mixmastamyk on Wednesday November 18, 2015 @02:34PM (#10590138)
> The Python 3 project needs to understand where it went wrong, acknowledge it...

Already happened years ago. Breaking backward-compat is the issue, and is a difficult situation for any popular platform. Guido has mention it won't happen again.

Re:Python 3 makes it easier to develop high-quality software by Semiapies on Wednesday November 18, 2015 @06:15PM (#10591452)
I'm not sure how to even read this, after I give it the requirements file for my last Flask project. It says six projects block transition for me, then offers me a goofy badge saying 4 blockers. Over on the right, the top thirteen of the projects in my file are black and not given as links.

(However, it is clear that going to Python 3 would not work.)

Re:Python 3 makes it easier to develop high-quality software by eeZi on Wednesday November 18, 2015 @12:34PM (#10589249)
Python 2 devs are also missing out on the amazing ecosystem developing around the new async features.

There are asyncio ports of many popular and less popular libraries, and it even has a nicer syntax than NodeJS.

Re:Python 3 makes it easier to develop high-quality software by shubhamjain on Wednesday November 18, 2015 @11:40AM (#10588791)
I don't believe its the same. Code from old version of PHP could still be expected to run the same in the newer version except for "deprecation" warnings maybe. Probably, a reason why it is not easy for PHP devs to fix all typecasting quirks their language has.

Python 3, on other hand, has tried to fix many design decisions and hence requires some transition effort. The effort is many folds more with so much software already written in Python 2 and large codebases. I am not surprised the adoption is slow.

Re:Python 3 makes it easier to develop high-quality software by frik on Wednesday November 18, 2015 @11:53AM (#10588901)
There were plans for a never released PHP6. Even some books were published for PHP6, it was meant to be the big unicode release. Most of its feature made it to the 5.4 version. In the end, it seems that it was the better decision to introduce new features in a smoother way. PHP hasn't suffered the problems that exists with Python 2 to 3. Although, there was the PHP4 to 5 transition that broke the older object oriented implemention - though most never used OO in PHP4 as it was known to be slow - so it wasn't a problem for the community.
Re:Python 3 makes it easier to develop high-quality software by PaulHoule on Wednesday November 18, 2015 @12:46PM (#10589342)
Note also that Sun/Oracle has periodically come out with updates for Java and these get taken up quickly, even by "enterprise" sorts of people. The enthusiasm for Java 8 is huge and the transition from 7 -> 8 is astonishingly trouble free and the 6 -> 7 transition was relatively bad because of the way in which JAXB creates POJOs changed in a way that broke people's assumptions.

Java has done a great job of evolving over time -- they made the controversial choice of using type erasure in generics. This makes generics much less useful than they are in C#, but Java kept compatibility between the classic and generic collections and C# didn't. Thus the C# code base has two different syntaxes for collections, particularly in the stuff that comes from Microsoft.

Re:Python 3 makes it easier to develop high-quality software by __jal on Wednesday November 18, 2015 @01:30PM (#10589704)
The cynical lesson I've taken from watching languages evolve over time is that, if you wish to provoke mass-migration to your new language version, having either a missing feature people want, or an annoying misfeature people don't, is really important.

Java and PHP have had both in spades. Back in the Perl 4 days, people really wanted a real object system[1].

Python 2 (aside from a few things that annoy the crap out of me, but apparently not many others) is a pretty complete, fairly decent language. Perl5 is pretty complete and people are pretty happy with it. (Well, the ones who write in it.)

[1] Yes, I know, not here to argue the semantics. It is "sufficiently OO" for what people want to do with it.

Re:Python 3 makes it easier to develop high-quality software by upofadown on Wednesday November 18, 2015 @11:35AM (#10588746)
You missed a whole subclass of people. Those that don't like Python3. Personally I believe Python3 took a wrong turn with the approach of converting everything to UTF-32 and making a big deal about the philosophical difference between "strings" and "bytes". I very much suspect the future will be something like how Julia deals with the issue, where UTF-8 can be just left as UTF-8.
Re:Python 3 makes it easier to develop high-quality software by mangecoeur on Wednesday November 18, 2015 @12:14PM (#10589071)
Apparently you don't deal with much non-ascii data if you don't like python3's string handling. It's literally the reason I first upgraded, because reading in simple text data on Python2 can break completely. Seeing as literally no language on earth can be written entirely in ascii characters (since even English text can contain words with accents or old fashioned things like the long s ſ) this is a pretty big issue.

Yes it is possibly to laboriously add code to check and encode/decode text, that might sometime be text and sometimes binary pretending to be text pretending to be binary. But why go through that pain when you could just use Python3 where things are clear and generally actually work.

Re:Python 3 makes it easier to develop high-quality software by eeZi on Wednesday November 18, 2015 @12:30PM (#10589216)
I fully agree. Python 3 makes life much, much easier with non-ASCII languages and prevents the annoying Unicode conversion bugs you see in most Python 2 code bases.

It's also very intuitive, which is great for teaching - I've always had a hard time explaining the difference between "strings" and "unicode" objects in Python 2, but the clear boundary makes it straight-forward in Python 3.

Re:Python 3 makes it easier to develop high-quality software by takeda on Wednesday November 18, 2015 @05:13PM (#10591158)
This is also significant reason why people have problem when migrating py2 code to py3. The code is actually buggy and mixes strings with bytes. Once they run it in Py3 all of the bugs show up because Py3 is stricter about it.

For the same reason if you need to write code that should work on Py2 and Py3, it's much easier to write it on Py3 and then add backward compatibility through "from __future__ import ..." and the six module.

Re:Python 3 makes it easier to develop high-quality software by eeZi on Friday November 20, 2015 @04:51PM (#10604110)
I've heard lots of good things about this project, by the way: http://python-future.org/

It's much less clunky than six. Instead of manually adding the compatibility layers to your code, it automates the process, which results in cleaner code. It can even translate (some) Python 2 modules on-the-fly.

Many projects switched to it from six.

Re:Python 3 makes it easier to develop high-quality software by SeanDav on Wednesday November 18, 2015 @03:19PM (#10590438)
Not always straight forward in Python 3. Try explaining to your class how to ignore case in string comparisons.
Re:Python 3 makes it easier to develop high-quality software by squeaky-clean on Wednesday November 18, 2015 @05:20PM (#10591193)
>Try explaining to your class how to ignore case in string comparisons

Okay: You can't. You'll need to convert both strings to the same case, and then compare those. In the majority of cases, you can do:

return string_a.lower() == string_b.lower()

But sometimes that won't work, because converting case isn't always easy. And that's a unicode issue, not a python issue. It's not even completely a unicode issue, just a "writing can be very weird" issue.

For example, "ß" (U+00DF) is a character that doesn't/shouldn't have an uppercase form. Unicode does support it as U+1E9E, but all the examples I know of will convert it to "SS" ([U+0053, U+0053]) instead.

Try this in a javascript console, for example.

  > "ß".toUpperCase().toLowerCase() === "ß".toLowerCase()
  false
  > "ß".toUpperCase() === "SS".toUpperCase()
  true
  > "ß".toLowerCase() === "SS".toLowerCase()
  false
  > "ß".length
  1
  > "ß".toLowerCase().length
  1
  > "ß".toUpperCase().length
  2
  > "ß".toUpperCase().toLowerCase()
  "ss"
Also, try ctrl+f'ing for "ss" on the Wiki page for ß [0] and see what it matches. Or ctrl+f this page for "ß" and it will match every "ss". I think it's pretty neat.

[0] https://en.wikipedia.org/wiki/%C3%9F

(edit: You might already know everything in this comment, but I started writing and it ended up being a cool exercise for myself, so I figured I'd submit).

Re:Python 3 makes it easier to develop high-quality software by mangecoeur on Thursday November 19, 2015 @05:05AM (#10593856)
> "writing can be very weird"

Absolutely! I think there's a lot of baggage in computing because it happened to emerge in places where language could mostly be written in simple characters with surprisingly simple rules. When exported internationally suddenly you deal with "oddities" that are really rather the norm in other languages.

In French for example is customary NOT to put accents on capital letters, even though it's possible. But then again sometime people do it anyway - it's not a hard rule. So often converting to lowercase may require contextual understanding because some words are written the same but with different accents, and would capitalize to the same string. Which word was meant would only be evident from the context of the sentence.

I suspect that if computing had rather developed somewhere where these things were commonplace we would have much better handling of it today.

Re:Python 3 makes it easier to develop high-quality software by throw_away on Thursday November 19, 2015 @01:10AM (#10593033)
Interesting that the "ss" search only works on safari and chrome, but not on firefox (which happens to be the only browser that lets you easily toggle case-matching AFAICT). I presume this means that the first two went with a toUpperCase() approach while FF went with toLowerCase(). I wonder which choice is better in the face of unicode type oddities like this one.
Re:Python 3 makes it easier to develop high-quality software by zo1 on Wednesday November 18, 2015 @03:34PM (#10590556)
That depends on the type of strings you are comparing. If you include the full Unicode set, you're going to run in to huge troubles in probably all languages simply due to the nature of the Unicode character set, rather than the specific language's handling of strings and Unicode code points.

But, personally, if you're teaching Python to a class, whether it's 2.x or 3.x. You should probably stick to using some variation of string.upper() == string.upper() to explain simple comparisons, due to simplicity.

Re:Python 3 makes it easier to develop high-quality software by c-oreills on Thursday November 19, 2015 @08:14AM (#10594529)
This is exactly what str.casefold is for: https://docs.python.org/3/library/stdtypes.html#str.casefold
Re:Python 3 makes it easier to develop high-quality software by SeanDav on Saturday November 21, 2015 @02:28PM (#10607732)
casefold is only part of the story and by itself does not supply a complete solution.
Re:Python 3 makes it easier to develop high-quality software by c-oreills on Monday November 23, 2015 @09:54AM (#10614866)
Could you expand on where it falls short? I'm interested, having not yet relied on it heavily.
Re:Python 3 makes it easier to develop high-quality software by krick on Wednesday November 18, 2015 @01:37PM (#10589756)
I would suspect that you don't deal with much non-ascii data if you like python3's string handling. In Python 2 you have control over encodings. You can work with unicode-strings internally, but you receive and produce bytes, which is actually the only truth in the computer world. Everything else is a lie. It would be nice to pretend that there's only 1 encoding in the world and that encoding is roughly equivalent to unicode itself, but this isn't true. World doesn't work on UTF-8. By abstracting encodings Python 3 obscures what's actually happening.
Re:Python 3 makes it easier to develop high-quality software by zo1 on Wednesday November 18, 2015 @03:38PM (#10590581)
Not at all, it forces you to unify to storing character data as Unicode code-points. Eventually if you go down far enough, you're reading the raw bytes you speak of from some sort of external system/file/DB. It's at that point that you define your encoding, and never worry about it every again. Except for when you touch those external systems/files again, again with full knowledge about what encoding is required.

Either way, if you're not dealing with differently encoded character data in Python 2.x as noted above, you're causing yourself some unnecessary headaches.

Re:Python 3 makes it easier to develop high-quality software by takeda on Wednesday November 18, 2015 @05:51PM (#10591343)
Something tells me you did not experience the pre-unicode world around 90s. I did, and I didn't like it. I prefer it is abstracted from me thankyouverymuch.

But seriously, all what python (and as a matter of the fact majority of languages) is placing a clear distinction between a byte and a character. This forces developers to keep in mind what they are dealing with and reduces number of bugs.

The truth is that if you treat everything as bytes, chances are that your program has bugs. Actually good test is to convert your existing Py2 code to Py3, if you have encoding/decoding errors then your code was buggy.

Re:Python 3 makes it easier to develop high-quality software by Avernar on Thursday November 19, 2015 @02:13AM (#10593229)
> But seriously, all what python (and as a matter of the fact majority of languages) is placing a clear distinction between a byte and a character.

No, they make a clear distinction between bytes and code points. This is the wrong approach as code points are not characters. People are going to think that everything is fine now that they're using code points and the unicode type when they should be using a unicode library for text manipulation.

Re:Python 3 makes it easier to develop high-quality software by vetinari on Wednesday November 18, 2015 @05:10PM (#10591145)
In Python2, I couldn't just print the string I just read from file - such were the joys of Windows console, which has different encoding than the rest of the system (cp85x in console, cp125x in filesystem & gui)...

Compared to that, Python3 was like gift from heaven.

Re:Python 3 makes it easier to develop high-quality software by yepguy on Wednesday November 18, 2015 @12:33PM (#10589235)
Python 3 is okay-ish for developers who don't think about encodings unless forced to, but it actually makes it more difficult to do "the right thing":

http://lucumr.pocoo.org/2014/5/12/everything-about-unicode/

Re:Python 3 makes it easier to develop high-quality software by vetinari on Wednesday November 18, 2015 @05:17PM (#10591178)
Having to thing about encoding all the time, without being forced to, brings in a lot of unwanted accidental complexity.

Did you know, that there is an operating system, where a simple loop that reads lines from file and prints them to the console will end up with errors? (that being Windows and some European code pages, where console is cp85x, filesystem read and write cp125x).

Re:Python 3 makes it easier to develop high-quality software by mark-r on Wednesday November 18, 2015 @06:21PM (#10591486)
The accidental complexity here is due to Windows, not Python. Python would rather generate an error than give you garbled output without your knowledge.
Re:Python 3 makes it easier to develop high-quality software by vetinari on Thursday November 19, 2015 @02:45AM (#10593335)
Most developers are not aware of that and write code, that just bombs out for the user. The user would prefer at least garbled text instead of exception, because he is used to that anyway in other apps. He has at least some result, instead of stack trace.

Well, and that's before the user tries to redirect stdout to file, in which case it will end up with error too, because that is going to be 7-bit ascii.

Anyway, Python3 does the right thing with defining the encoding at the edges and everything works right for the user out of the box.

Re:Python 3 makes it easier to develop high-quality software by maxerickson on Wednesday November 18, 2015 @11:46AM (#10588854)
Recent versions only convert to UTF-32 when a wide character is present (they also use 2 byte characters when that is sufficient):

https://docs.python.org/3.3/whatsnew/3.3.html#pep-393-flexib...

Glancing at Julia, I'm not sure "invalid character index" is really a great thing to have to handle when dealing with text. It does seem that systems for dealing with text will eventually offer various apis beyond just a somewhat disfunctional character api (which I think is a fair description of str in Python 3).

Re:Python 3 makes it easier to develop high-quality software by Avernar on Wednesday November 18, 2015 @03:25PM (#10590483)
> Recent versions only convert to UTF-32 when a wide character is present

It's non-BMP characters that trigger the conversion to 4 bytes per codepoint. Unfortunately Emoji are non-BMP and are super common.

Re:Python 3 makes it easier to develop high-quality software by rspeer on Wednesday November 18, 2015 @04:35PM (#10590916)
If you're dealing with emoji -- and yes, they are incredibly common -- you'd better not use any Python before 3.3.

Non-BMP characters in 2.x-3.2 give you implementation-specific behavior. It can vary from one system to the next. Some string methods fail on "narrow builds" of Python (the ones that store their characters in UCS-2), while others return different results than they would on a "wide build".

Re:Python 3 makes it easier to develop high-quality software by Avernar on Wednesday November 18, 2015 @04:48PM (#10591008)
Not an issue because I don't use the Unicode type. All my data comes in as UTF-8 on which I do a quick validation check. Then it's just handled as bytes all the way to and from my database.
Re:Python 3 makes it easier to develop high-quality software by rspeer on Thursday November 19, 2015 @03:04AM (#10593397)
Sure, that is an option that's always available. You don't have to worry if Python's string methods are wrong if you're opting not to use them.
Re:Python 3 makes it easier to develop high-quality software by upofadown on Wednesday November 18, 2015 @11:55AM (#10588919)
>Recent versions only convert to UTF-32 when a wide character is present (they also use 2 byte characters when that is sufficient) ...

I happen to know this fact. Are you really asking that everyone that comments on a language acknowledge all the possible implementation details? I at no point complained about the memory required to hold strings in CPython so I can't really understand why you would bring this up.

Re:Python 3 makes it easier to develop high-quality software by maxerickson on Wednesday November 18, 2015 @12:00PM (#10588963)
Because someone else reading "converting everything to UTF-32" might not be aware of the nuance. That's all.
Re:Python 3 makes it easier to develop high-quality software by teach on Wednesday November 18, 2015 @12:24PM (#10589174)
I'm a casual user of Python 2, and I certainly didn't know this, so I appreciated the clarification.
Re:Python 3 makes it easier to develop high-quality software by upofadown on Wednesday November 18, 2015 @05:56PM (#10591363)
Sorry for the snark but this has become a trigger for me. Every time anyone disagrees with the way Py3 does Unicode someone will immediately assume that that person is just confused about the implementation.
Re:Python 3 makes it easier to develop high-quality software by Avernar on Wednesday November 18, 2015 @03:19PM (#10590435)
I agree. Everything is so much simpler when you use UTF-8 everywhere. Being effectively forced to use the ASCII/UCS2/UCS4 unicode type in Python3 is the only thing that has kept me from switching to it.
Re:Python 3 makes it easier to develop high-quality software by 1st1 on Wednesday November 18, 2015 @04:15PM (#10590807)
The only thing you are forced to think about is whether you're working with strings or with bytes. Unicode is handled transparently (but you do need to know the encoding when you're decoding bytes). How it's stored internally is none of your concern.
Re:Python 3 makes it easier to develop high-quality software by Avernar on Wednesday November 18, 2015 @04:32PM (#10590903)
Yes it is. Every conversion to and from the unicode type and every expansion of the unicode type from 1 to 2 to 4 bytes eats up processor time, cache/memory bandwidth and puts more pressure on the garbage collector.

And I don't want to be forced to distinguish between strings and bytes. To me they are one and the same as I use UTF-8 for everything. Not once in my code do I need to encode/decode anything.

Re:Python 3 makes it easier to develop high-quality software by Avernar on Thursday November 19, 2015 @01:26AM (#10593089)
Nice, down voted for someone not agreeing with me. I'd rather know what you though was wrong with what I wrote.

I've always wanted to know how things are done internally. Makes me a better programmer. What I said about all this extra conversion does to performance is correct, especially when scaling up.

And my choice to use UTF-8 internally is a valid way to do things. I make sure that what comes in from the outside world is UTF-8 (either by rejecting or converting) and then don't have to worry about it from that point on. The bulk of my code deals with outbound data so no need for me to do any encoding as it's already UTF-8.

Re:Python 3 makes it easier to develop high-quality software by mark-r on Wednesday November 18, 2015 @06:17PM (#10591459)
Most of us aren't so lucky. I really do wish we lived in a 100% UTF-8 world. But even if we did, I don't see how e.g. regular expressions would work.
Re:Python 3 makes it easier to develop high-quality software by Avernar on Thursday November 19, 2015 @01:13AM (#10593044)
The world seems to be headed to UTF-8 for everything. That's why having Python3 force us away from it doesn't make sense to me.

As for regular expressions, they work perfectly if your delimiters are in the ASCII range. If you're trying to match a single character then they don't work for Unicode even when using Python3's unicode type. The reason is that code points are not characters.

Re:Python 3 makes it easier to develop high-quality software by 1st1 on Thursday November 19, 2015 @10:13AM (#10595173)
UTF-8 is just an encoding. Unicode is a more general term. Python 3 allows you to convert bytes to unicode from any encoding, including utf-8 (which is default). Using utf-8 as the internal unicode representation is flawed too - index and length operations are slow.

For regexes, \w matches any unicode word character (unless ASCII mode is on), just use it on strings. If you use it on bytes, then yes, you can only match bytes.

Re:Python 3 makes it easier to develop high-quality software by Avernar on Thursday November 19, 2015 @01:35PM (#10596570)
> Using utf-8 as the internal unicode representation is flawed too - index and length operations are slow.

And this perfectly illustrates what I posted in another comment elsewhere. Thinking that doing operations on code points (indexing and length) is the proper thing to do. You need to use a unicode library to work on graphemes and take into account grapheme clusters.

> For regexes, \w matches any unicode word character

From what I've read, \w just matches code points. I did find that the https://pypi.python.org/pypi/regex module supports \X which matches graphemes as per the unicode spec http://www.unicode.org/reports/tr29/.

Re:Python 3 makes it easier to develop high-quality software by msellout on Wednesday November 18, 2015 @12:41PM (#10589303)
I think we're going to see the switch occur quickly now. This year I've seen several large enterprise projects using Python 3. Also, the release of v3.5 brings useful features that will not be backported as 2.8.
Re:Python 3 makes it easier to develop high-quality software by incepted on Wednesday November 18, 2015 @01:40PM (#10589777)
It's unproven because nobody uses it and nobody uses it because it's unproven.
Re:Python 3 makes it easier to develop high-quality software by GFK_of_xmaspast on Wednesday November 18, 2015 @11:34AM (#10588743)
I don't have any compelling reason to move off of python 2.7, and for my use cases the positives are outweighed by the negatives (in particular getting used to needless changes).
Re:Python 3 makes it easier to develop high-quality software by strictnein on Wednesday November 18, 2015 @12:23PM (#10589154)
Speaking of Node, it still requires Python 2.7, right? Or at least node-gyp does: https://github.com/nodejs/node-gyp
Re:Python 3 makes it easier to develop high-quality software by gsnedders on Wednesday November 18, 2015 @12:40PM (#10589297)
gyp itself still relies on Python 2 (and, to my knowledge, Google has mostly moved on gyp, so they're unlikely to do anything large to it).