Re:Python Patterns - An Optimization Anecdote
by edanm on Tuesday November 02, 2010 @09:34AM
(#1860328)
First off, I'm really no expert in Python or optimizations - I'm only reporting on observations.
Having said that, I was pretty surprised that the list comprehensions ran faster than map, exactly because of point 2 - I had assumed that map used a c-loop, whereas list comprehensions did not. Unfortunately, I don't have that code sitting around anywhere so I can't redo the tests - I just remember being very surprised by those results.
Re:Python Patterns - An Optimization Anecdote
by deathflute on Tuesday November 02, 2010 @09:57AM
(#1860400)
In my experience edanm is correct. map/reduce/filter are faster with built-in functions, but list comprehensions are faster with user-defined ones
For eg. using python 2.5.2
In [9]: l = ['10'] * 5000000
In [10]: timeit -n5 map(int, l)
5 loops, best of 3: 1.29 s per loop
In [11]: timeit -n5 [int(i) for i in l]
5 loops, best of 3: 1.64 s per loop
Also, and this is just on a more minor, pedantic note, list comprehensions basically are regular for loops, except with the use of the list-append opcode instead of a getattr-call pair. This produces a minor speedup, but it's nothing to write home about, honest. (using a different pastebin because it has a feature I needed): http://ideone.com/dT1LG . The story is a bit different on 3.x, where list comps are implemented very differently: http://ideone.com/58FGZ