(filter (map expensive_call X) cond)
[y for y in [expensive_call(x) for x in X] if cond(y)]
Incidentally, this isn't even an issue in a pure functional language with sufficiently smart compiler.
Plus, you get 5 mentions (3 y's and 2 x's) of some intermediate variables instead of 0 in your code, so both token-wise and char-wise, map/filter alternative is shorter, and less a mental burden (think about the "succinct" idea by PG).
If you can put together a nontrivial usage of map+filter with at least three source collections that's more concise than the equivalent list comprehension I'll (figuratively) eat my hat.
map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l}
filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s),
product(widgets, sprockets, locations)))
The argument in favor of set-notation being higher level is it's less specific (it doesn't explicitly provide a sequence of operations, just an outcome).
List comprehensions look like set notation but have an implicit procedural translation you have to keep in mind to use them well, so it's a toss-up.
I prefer map/filter/reduce when sequencing has large performance implications but for simple filtering or raw-data-shaping comprehensions read more smoothly.
Now you know why many people like me are gradually pissed off by Python and start the exile ... currently trying Scala and it seems a nice language (with list comprehension too! :)
> List comprehensions look like set notation but have an implicit procedural translation you have to keep in mind to use them well, so it's a toss-up.
Actually I think that's the problem I have with list comprehensions: I use them a lot in my code, usually 1~3 levels nested and then have a hard time tracking down the order of implicit loops (which is inner vs outer) and make sure the intermediate variables (x for x in y for y in z ...) do not clash ... OK maybe I'm using it too much and in the wrong way :(
> I prefer map/filter/reduce when sequencing has large performance implications but for simple filtering or raw-data-shaping comprehensions read more smoothly.
I didn't know map/filter is faster than list comprehensions? I thought both are optimized by Python interpreter. But I like the idea of knowing that at least map can be parallelized easily. But since Python does not utilize multicore in a decent way, all bets are off :(
do { w <- widgets;
s <- sprockets;
l <- locations;
guard (l `hasInStock` w);
guard (l `hasInStock` s);
guard (w `isUsableWith` s);
return (w, s, l); }
If eg I edited it to be:
do { l <- locations;
w <- widgets;
guard (l `hasInStock` w);
s <- sprockets;
guard (l `hasInStock` s);
guard (w `isUsableWith` s);
return (w,s,l); }
List comprehensions: the set contains this.
Map/filter: apply these functions to the set.
Procedural: follow this recipe for turning one list into another.
[fn(x) for x in xs for xs in xss if cond(xs)]
filtered = [x for x in seq if x>10]
For simple cases, yes. But I wouldn't say
[each for each in [expensive_call(x) for x in seq] if cond(each)]
filter(cond, map(expensive_call, seq))
Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python.
Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code).
(disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)
Later, I ran into some cases where I wanted a multi-line lambda. And my thought was "aaragh134!#?!"
Then, a weird thing happened. I started making a conscious effort to follow PEP 8. 79 column limit? Seriously? That sucks. But after weeks of struggling with it, something finally hit me. I realized I was writing better code by forcing myself to reduce code density. Sure, it was a little bit longer, but I spend way more time reading it than writing it.
Stop trying to fight it; assign a name to that lambda. Readability counts.
Stop trying to be clever; assign a name to that inner expression. Flat is better than nested.
De-emphasizing functional operations makes it more likely for libraries to work in a non-functional style, for tutorials to do so, etc.
It's tiring fighting against a community and a (benevolent) dictator that disagree with you.
That summarizes the problem I guess :D
Actually only one line of code is enough for each of map/filter:
def map(fn, seq): return [fn(each) for each in seq]
def filter(cond: seq): return [each for each in seq if cond(each)]
(filter #(> x 10) seq)
I would like to point out, however, that CL allows you to write:
(loop for x in seq
when (> x 10)
collect x)
(loop for i in *random*
counting (evenp i) into evens
counting (oddp i) into odds
summing i into total
maximizing i into max
minimizing i into min
finally (return (list min max total evens odds)))
I switch between map/filter and loop depending on whether I'm working with predefined functions (e.g., (filter 'less-than-ten seq)), handling multiple sequences, doing side-effects, etc.
CL could really use something like Python's generators though. The loop macro is an ugly hack in comparison.
The question is whether "Pythonic," as the community defines it today, is optimal in all cases. As an analogue, there have been enough talks about things the Java community considers to be stylistically optimal that look really horrible compared to implementations in other languages -- like Python. :-) Pythonic style should be a guide, not an edict, and should be deviated from or redefined when it makes sense. The examples riobard provided already show the syntax weighing things down, a situation where syntax should give way to a more functional style approach.
As for it creating a cleaner language, I try to approach this, as with all things, from the perspective of being an language-agnostic programmer. From that perspective, I do like the python syntax for simple things like what you defined, but under heavier weight mapping and filtering operations, the map/filter function call syntax seems a lot cleaner. There's nothing wrong with syntactic sugar, but I would assert that it will tend to suck when it is all you have.
You can arguably trace Python's list comprehension syntax all the way back to setl, a "set-theoretic programming language", and the resemblance to mathematical set-builder notation is intentional; compare
- let B = { g(a) s.t. | a \in A and f(a) holds}
- B = [g(a) for a in A if f(a)]
If you're used to thinking in sets having to decompose into "maps" and "filters" is a speedbump; easy to do but nice to avoid.
Where list comprehensions really start to shine is making it comparatively trivial to pull from multiple source collections without a lot of ugly machinery:
[{'widget':w,'sprocket':s,'location':l} for w in widgets for s in sprockets for l in locations if l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s)]
map(lambda i: {'widget':i[0], 'sprocket':i[1], 'location':i[2]}, filter(lambda i: i[2].hasInStock(i[0]) and i[2].hasInStock(i[1]) and i[0].isUsableWith(i[1]), itertools.product(widgets,sprockets,locations)))
It gets even worse if you try to be clever with the sequence of operations.
You might look at that definition and say lo! I can pre-filter out stuff not in stock at each location and make things more efficient. Naively you'd wind up with:
map(lambda d: {'location':d[0], 'widget':d[1], 'sprocket':d[2]}, flatten(map(lambda l: list(filter(lambda p: p[1].isUsableWith(p[2]), itertools.product([l],filter(lambda w: l.hasInStock(w), widgets), filter(lambda s: l.hasInStock(s), sprockets)))), locations))) #nb must-supply-you-own-flatten-method
[{'widget':w,'sprocket':s,'location':l} for l in locations for w in [widget for widget in widgets if l.hasInStock(widget)] for s in [sprocket for sprocket in sprockets if l.hasInStock(sprocket)] if w.isUsableWith(s)]
Level 0: for-loops with an explicit accumulator
Level 1: map + filter (!)
Level 2: ??? arguably an atemporal set-theoretic approach
In practice in python list comprehensions are a superior syntax for computing with multiple source collections.
(!) Really all you need is reduce
map = lambda f,l: reduce(lambda h,t: h + f(t), l, [])
filter = lambda f,l: reduce(lambda h,t: h + t if f(t) else h, l, [])
A saner map/filter/product version:
map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l}
filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s),
product(widgets, sprockets, locations)))
e.g.
equals but equals note because of "expensive_call", it's inefficient (and silly) to do So map/filter combination gives more flexibility than list comprehension, and for functional-thinking minds, it's just so natural to think in abstract terms of passing functions around. List comprehension is pretty syntactical sugar to do similar things, but it forces you think about the "how to do" instead of "what to do".That said, it's not really "compelling" though -- there is no real "compelling" reason to switch from one Turing-complete language to another given that you can do the same thing eventually. But hey, it's the itches that drive us nuts, isn't it? :)
Just my 2 cents