News for hackers. Stuff that matters.
Re:Why Python Runs Slow, Part 1: Data Structures by erbdex on Thursday February 13, 2014 @07:52AM (#7231008)
"My experience in working on Graphite has reaffirmed a belief of mine that scalability has very little to do with low-level performance but instead is a product of overall design. I have run into many bottlenecks along the way but each time I look for improvements in design rather than speed-ups in performance. I have been asked many times why I wrote Graphite in Python rather than Java or C++, and my response is always that I have yet to come across a true need for the performance that another language could offer. In [Knu74], Donald Knuth famously said that premature optimization is the root of all evil. As long as we assume that our code will continue to evolve in non-trivial ways then all optimization6 is in some sense premature.

For example, when I first wrote whisper I was convinced that it would have to be rewritten in C for speed and that my Python implementation would only serve as a prototype. If I weren't under a time-crunch I very well may have skipped the Python implementation entirely. It turns out however that I/O is a bottleneck so much earlier than CPU that the lesser efficiency of Python hardly matters at all in practice."

-- Chris Davis, ex Google, Graphite creator

Re:Why Python Runs Slow, Part 1: Data Structures by jnbiche on Thursday February 13, 2014 @10:20AM (#7231771)
This is the key. Very often, the app is IO bound, and moving to C will make little difference.

This is one reason why I'm very excited about Python's new AsyncIO module in Python 3 only. It's asyncore done right, and is a great way to write network applications. I look forward to seeing what is built on top.

Re:Why Python Runs Slow, Part 1: Data Structures by mess on Thursday February 13, 2014 @02:18PM (#7233548)
Would this work with existing libraries? For example, if you want to use whisper without blocking the thread.
Re:Why Python Runs Slow, Part 1: Data Structures by a8da6b0c91d on Thursday February 13, 2014 @02:43PM (#7233739)
> Very often, the app is IO bound

I hear this all the time but it really doesn't square up with my personal experiences. I've had to re-implement dynamic language stuff in C++ many times to get acceptable performance. No real design changes, just straight ports to C++. The entire value proposition of Golang is that the scripting language dynamic type systems are too slow, and it seems like loads of folks agree.

Re:Why Python Runs Slow, Part 1: Data Structures by pjmlp on Thursday February 13, 2014 @04:07PM (#7234352)
That is why they never used Dylan, Lisp, SELF or any other dynamic language with native compilers.
Re:Why Python Runs Slow, Part 1: Data Structures by a8da6b0c91d on Thursday February 13, 2014 @04:15PM (#7234401)
Those dynamic languages are only performant when you add a bunch of type information. You wind up writing a simulacrum of C and it still under-performs.
Re:Why Python Runs Slow, Part 1: Data Structures by lispm on Thursday February 13, 2014 @05:06PM (#7234787)
The idea is that there is some useful speed for much of the application by using native compilers. Higher speed might be needed only in parts of the program. Then a type inferencer, etc. comes to use. If that's still not enough one calls C or assembler routines - but, say, 95% of the application can be written in Lisp.
Re:Why Python Runs Slow, Part 1: Data Structures by erbdex on Friday February 14, 2014 @07:21AM (#7238122)
> I've had to re-implement dynamic language stuff in C++ many times to get acceptable performance.

Performance in a certain sense isn't just speed of a single uncoupled module, but also the end-to-end seamlessness of a system. The higher level languages might 'waste' 300ms in every function call, but save perhaps years in development time.

Also- architecting and developing medium-sized projects is not just the 'ENTERPRISE THING' anymore. A vastly greater number of people now have a-shot-at larger projects. So this creative freedom and technical UN-chaining also add to performance in a very in-direct way by allowing better minds to enter the game.

Re:Why Python Runs Slow, Part 1: Data Structures by a8da6b0c91d on Friday February 14, 2014 @09:14AM (#7238564)
This is all true, but the endlessly repeated line about "IO Bound" needs to die. It's just not true. Most stuff written in dynamic languages is a lot slower than it would be in a statically typed language.