- Python has easy development (https://xkcd.com/353/)
- Great libraries (ie, free matlab)
- Cython for efficiency via C
- The algorithms themselves determine speediness (ie numerical methods)
This is so important I wish people would focus more on it. I recently rewrote some Javascript code in (pure) python and got a good 2 orders of magnitude speed up on large inputs just by picking the right data structures and replacing an O(n^3) nested loop with an O(n log n) approach.
We do scientific computing at my company. Numpy does 90% of the work, but there are some algorithms that just aren't easily expressed with arrays. That's where Cython comes in.
Numpy and scipy have been the core of a huge amount of my optimisations. The first question I try and ask is
"Could this be solved with matrix multiplications and summing?"
Often the answer is "yes" and allows you to group a huge amount of calculations all together, and use the heavily optimised code available numpy/scipy.
I recently swapped out something that was running at about 100 rows calculated/second to about half a million in about 0.2s.
for k in dict.keys():
...
then python first builds a list of all the keys, loops through them and then throws away the list. If the dict is large, this can be quite expensive. The correct way is to either use .iterkeys() which returns an iterator which generates the keys one at a time, or simply iterate directly over the dict, saving you need to first copy all the keys into a list you'll just throw away.This has been 'fixed' in python3 and .keys() now returns an iterable view of the keys, and if you actually want a list of the keys you have to explicit and write list(dict.keys())
for k in dict:
...
if k in dict:
...
If you do need a list of keys, list(dict) has the advantage of working in both Python 2 and 3.The equivalent python2 behaviour can be obtained using list(somedict.keys())
I tried a very simple toy program the other day and while I had to write some things slightly un-pythonically (it can't deal with syntax like a[:] = b+c yet), it performed practically as good as hand-written C code.
Java is compiled to bytecode, but it isn't a "compiled language" since that bytecode has to be interpreted by the JVM. All the good libraries the author mention are probably implemented in C or FORTRAN, and so a true implementation in C with the right compiler optimizations would for sure be faster than the python code.
True. But this assumes that time is not a constraint.
I think you need to think of it this way (as a thought experiment): you start two programmers off, one in C and one in Python, both with a vague understanding of how to solve the problem and approximately the same skill level. Then after X hours, you stop both and see how far they've gotten.
I think the argument then is that you might find that the C programmer hasn't yet solved the problem, whereas the Python programmer might have a solution + a deeper understanding of where the slow parts are, and started optimizing those.
So in the end the C programmer may win, but perhaps it's not about winning in this sense but more about how fast you can get to a point where you can move on to the next thing.
I think as developers we often underestimate the amount of work we need to put into the non coding part of development for larger projects, like testing, debugging, optimization, and design. When these components become important then despite the local productivity gains of using rapid development languages like python or Julia, it would overall be a wash if you're writing the system from scratch in a language like C.
[0]: https://www-01.ibm.com/support/knowledgecenter/SSYKE2_7.0.0/... [1]: https://www.azul.com/products/zing/
Finally, you can release the GIL with Cython: http://docs.cython.org/src/userguide/parallelism.html
"The end result is an implementation several orders of magnitude faster than the current reference implementation in Java. ... [Python] makes the first version easy to implement and provides plenty of powerful tools for optimization later when you understand where and how you need it." [edited to be a statement instead of rhetorical question]