News for hackers. Stuff that matters.
Re:Python 3.11 vs 3.10 performance by bilsbie on Wednesday July 06, 2022 @11:47AM (#32002675)
When is that coming? Yes the error messages are confusing in a big list comprehension.
Re:Python 3.11 vs 3.10 performance by alexfromapex on Wednesday July 06, 2022 @12:17PM (#32003114)
Re:Python 3.11 vs 3.10 performance by teekert on Wednesday July 06, 2022 @01:15PM (#32003934)
Whenever I nest a list comprehension I feel dirty anyway, I wonder if a better error message will help solve something that is just hard to grasp sometimes.
Re:Python 3.11 vs 3.10 performance by 8organicbits on Wednesday July 06, 2022 @01:47PM (#32004376)
"Debugging is twice as hard as writing the code in the first place. Therfore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Rajanand

Although, personally, I enjoy python list comprehensions.

Re:Python 3.11 vs 3.10 performance by timoteostewart on Wednesday July 06, 2022 @01:55PM (#32004479)
I believe that quote about debugging being twice as hard as coding is by Brian Kernighan.

https://en.m.wikiquote.org/wiki/Brian_Kernighan

Re:Python 3.11 vs 3.10 performance by 8organicbits on Wednesday July 06, 2022 @02:21PM (#32004774)
Maybe, I grabbed the first matching quote I found. I can't attribute it on my own.
Re:Python 3.11 vs 3.10 performance by timoteostewart on Wednesday July 06, 2022 @03:00PM (#32005302)
No sweat. I’m a quote nerd so it jumped out at me
Re:Python 3.11 vs 3.10 performance by mikepurvis on Wednesday July 06, 2022 @02:14PM (#32004711)
Not even with nesting— as soon as it's long enough that I'm breaking it into multiple lines, I'm immediately like, okay this is long enough to just be a normal for-loop, or maybe a generator function. But it doesn't need to be a list comprehension any more.
Re:Python 3.11 vs 3.10 performance by polski-g on Wednesday July 06, 2022 @01:27PM (#32004088)
All list comprehensions should error with message "FATAL PARSE ERROR: Will you really understand this in six months?"
Re:Python 3.11 vs 3.10 performance by teekert on Thursday July 07, 2022 @04:52AM (#32011486)
Something like this I find ok (just a filter):

    [x for x in y if x is not z]
Sometimes though, facilitated by Jupyter's notebooks making executing as you build the code very easy, I create a beast like this:

    [os.path.join([x for x in y if x is not z]) + '/bin' for a in b if 'temp' not in a]
Yes this is fictional and probably contains an error but you get the point, you can filter lists of lists like this, but it's really unfriendly to anyone trying to understand it later.
Re:Python 3.11 vs 3.10 performance by wheelerof4te on Wednesday July 06, 2022 @03:11PM (#32005422)
List comprehensions should be used for making quick lists that contain simple calculations or filters.

Anything more will come back to bite you later.

Re:Python 3.11 vs 3.10 performance by TremendousJudge on Wednesday July 06, 2022 @01:42PM (#32004290)
If people complain about Rust's borrow checker already...
Re:Python 3.11 vs 3.10 performance by psalminen on Wednesday July 06, 2022 @01:49PM (#32004398)
Big list comprehensions cause confusion in many ways. KISS is crucial with them.
Re:Python 3.11 vs 3.10 performance by mcronce on Wednesday July 06, 2022 @02:08PM (#32004647)
I agree. The rule of thumb I follow is that if the list comp doesn't fit on one line, whatever I'm doing is probably too complicated for a list comp.
Re:Python 3.11 vs 3.10 performance by gpderetta on Thursday July 07, 2022 @06:14AM (#32012058)
But why? The only reason is that error messages are awful and intermediate values can't be named. There is in principle no reason why list comprehension need to be worse than straight line for loops. The deciding factor to chose betwee the two should be whether I'm writing something for side effects as opposed to for its value.
Re:Python 3.11 vs 3.10 performance by mcronce on Saturday July 09, 2022 @02:49AM (#32032530)
You just listed two very good reasons: Intermediate values can't be named, and error messages are awful.
Re:Python 3.11 vs 3.10 performance by agumonkey on Wednesday July 06, 2022 @01:29PM (#32004114)
In a way it's a nice incentive not to nest too much (I have that tendency too)
Re:Python 3.11 vs 3.10 performance by yantrams on Wednesday July 06, 2022 @02:04PM (#32004604)
Re:Python 3.11 vs 3.10 performance by ChadNauseam on Wednesday July 06, 2022 @02:38PM (#32005033)
Hold mine :D https://github.com/anchpop/genomics_viz/blob/master/genomics...

That's one expression because it used to be part of a giant comprehension, but I moved it into a function for a bit more readability. I'm considering moving it back just for kicks though.

My philosophy is: if you're only barely smart enough to code it, you aren't smart enough to debug it. Therefore, you should code at your limit, to force yourself to get smarter while debugging

Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @04:10PM (#32006163)
Yours is nice and readable. Parents' one is not, but it feels like the indentation is deliberately confusing. I'd lay it out like so:

   tags = list(set([
     nel
     for subli in [
       mel
       for subl in [
         [[jel.split('/')[2:] for jel in el] 
         for el in classified
       ]
       for mel in subl 
     ]
     for nel in subli
     if nel
   ]))
Still not very readable, tho. But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that doesn't scale at all.

Side note: one other thing I always hated about those things in Python is that there's no way to bind an intermediary computation to a variable. In C# LINQ, you can do things like:

   from x in xs
   let y = x.ComputeSomething()
   where y.IsFoo && y.IsBar
   select y
In Python, you have to either invoke ComputeSomething twice, or hack "for" to work like "let" by wrapping the bound value in a container:

   y
   for x in xs
   for y in [x.ComputeSomething()]
   if y.IsFoo and y.IsBar
It's not just about not repeating yourself or not running the same code twice, either - named variables are themselves a form of self-documenting code, and a sequence pipeline using them even where they aren't strictly needed can be much more readable.
Re:Python 3.11 vs 3.10 performance by dragonwriter on Wednesday July 06, 2022 @10:20PM (#32009272)
> But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that doesn't scale at all.

Choices like wrapping a set constructor around a list comprehension rather than just using a set comprehension don't help, neither does using multiple nested no-transformation comprehensions laid out that way.

By hand, on mobile, I think this reduces to (assuming “from itertools import chain” in the file header):

   tags = list({
     nel
     for subli in chain.from_iterables(chain(
       [[jel.split('/')[2:] for jel in el] 
       for el in classified))
     for nel in subli
     if nel
   })
Re:Python 3.11 vs 3.10 performance by andrewaylett on Thursday July 07, 2022 @10:56AM (#32014904)
Since it's all generator expressions internally anyway, an intermediate doesn't actually add appreciable (any?) runtime overhead but does make the code more readable:

    ys = (x.ComputeSomething() for x in xs)
    result = [y for y in ys if y.isFoo and y.isBar]
Re:Python 3.11 vs 3.10 performance by texaslonghorn5 on Wednesday July 06, 2022 @04:15PM (#32006225)
Does the new walrus := in python solve your problem?
Re:Python 3.11 vs 3.10 performance by roelschroeven on Wednesday July 06, 2022 @06:29PM (#32007685)
The official name is assignment expression, which can be good to know to find documentation. Here's the PEP: https://peps.python.org/pep-0572/
Re:Python 3.11 vs 3.10 performance by otherme123 on Thursday July 07, 2022 @01:57AM (#32010407)
Curiously, searching the docs for "walrus" (https://docs.python.org/3/search.html?q=walrus) yields exactly the results you need, but searching for "assignment expression" yields the previous results mixed a lot of other results due to the word "expression".
Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @09:25PM (#32008944)
Not really; where would you put it inside the []?
Re:Python 3.11 vs 3.10 performance by dragonwriter on Wednesday July 06, 2022 @09:33PM (#32008981)
In the example from the end of the earlier post:

   y
   for x in xs
   if (y:=x.ComputeSomething()).IsFoo and y.IsBar
Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @11:20PM (#32009574)
I suppose that works, although readability is not great, to put it mildly.
Re:Python 3.11 vs 3.10 performance by texaslonghorn5 on Wednesday July 06, 2022 @11:21PM (#32009580)
Much better would be

[(y:=x.ComputeSomething()) for x in xs if y.IsFoo and y.IsBar]

Re:Python 3.11 vs 3.10 performance by int_19h on Thursday July 07, 2022 @03:01AM (#32010755)
That doesn't work, because that first subexpression is evaluated last, after "if".
Re:Python 3.11 vs 3.10 performance by ChadNauseam on Wednesday July 06, 2022 @11:37PM (#32009649)
For another example of how to use the walrus operator, I use it multiple times in the code I linked a few comments up
Re:Python 3.11 vs 3.10 performance by teddyh on Thursday July 07, 2022 @07:52AM (#32012807)
Re:Python 3.11 vs 3.10 performance by agumonkey on Wednesday July 06, 2022 @04:02PM (#32006055)
That code is fine in my book, there's more static code than for-comp and it's at work 3 level deep with no hard coupling between levels. Also well spaced and worded.
Re:Python 3.11 vs 3.10 performance by wvh on Wednesday July 06, 2022 @03:12PM (#32005438)
I don't think this is bad code. Maybe not the most Pythonic in the imperative sense, but certainly you could come across such constructs in languages that are more oriented towards functional programming. This could also be solved in a more readable manner by having a generator pipeline, though it would be a good idea to see what the performance of chained generators is like in your version and flavour of Python.
Re:Python 3.11 vs 3.10 performance by kzrdude on Wednesday July 06, 2022 @07:26PM (#32008147)
It's “not even” using for nesting in list comprehensions. This kind of thing:

   [x for stop in range(5) for x in range(stop)]