News for hackers. Stuff that matters.
Re:Python 3.11 vs 3.10 performance by alexfromapex on Wednesday July 06, 2022 @11:39AM (#32002566)
One of the biggest features I'm looking forward to is the more specific error messages. I can't tell you how much time I've wasted with cryptic errors that just point to a line that has a list comprehension or similar.
Re:Python 3.11 vs 3.10 performance by CobrastanJorji on Wednesday July 06, 2022 @02:42PM (#32005086)
I've been learning Rust recently. There are a number of things about the language that I dislike, but its error messages are an absolute joy. They clearly put an incredible amount of effort into them, and it really shows.
Re:Python 3.11 vs 3.10 performance by jmcgough on Wednesday July 06, 2022 @04:23PM (#32006339)
I wrote 2-3 of those and I really wish more languages had a similar approach with their standard library.

When you get an error there's a verbose explanation you can ask for, which describes the problem, gives example code and suggests how you can fix it. The language has a longer ramp-up period because it contains new paradigms, so little touches like this help a lot in onboarding new devs.

Re:Python 3.11 vs 3.10 performance by HdS84 on Thursday July 07, 2022 @12:57PM (#32016441)
I am a hug fan of ravendb's founder ayende. He always stresses how useful good error messages are, especially for self service. And he is certainly on spot with that, e.g "could not execute Mode None" is much less helpful than " environment variable X not set, could not determine Mode. Please set this as MODE=Agressive or MODE=Passive".
Re:Python 3.11 vs 3.10 performance by dralley on Wednesday July 06, 2022 @06:06PM (#32007422)
The exception is std::io::Error, which doesn't provide any information about which file failed to be opened/closed/read/whatever. I know this is because doing so would require an allocation, but it's still painful.
Re:Python 3.11 vs 3.10 performance by tialaramex on Wednesday July 06, 2022 @06:46PM (#32007853)
Your parent is thinking about the compiler errors, whereas std::io::Error (which I'm guessing you mean) is an Error type your software can get from calling I/O functions at runtime.

To be fair, if decorating the error with information about a filename is what you needed, since Rust's Error types are just types nothing stops you making your function's Error type be the tuple (std::io::Error, &str) if you have a string reference, or (though this seems terribly inappropriate in production code) leaking a Box to make a static reference from a string whose lifetime isn't sufficient.

Re:Python 3.11 vs 3.10 performance by estebank on Wednesday July 06, 2022 @06:40PM (#32007792)
The GP comment is referring to the compile time diagnostics and not the runtime diagnostics, which certainly could do with some out-of-the-box improvements but that you can extend in your own code (as opposed to compile errors which crate authors have little control over, at least for now).
Re:Python 3.11 vs 3.10 performance by rowanG077 on Wednesday July 06, 2022 @06:40PM (#32007797)
huh a stack trace should tell you that.
Re:Python 3.11 vs 3.10 performance by dec0dedab0de on Wednesday July 06, 2022 @02:39PM (#32005035)
This is the first time I ever heard someone complain about Python's error messages in the 10+ years I've been using it. Even people who just learned it, pick up reading tracebacks after about a day of practice. The only problem I ever see is if a library swallows too much, and gives a generic error, but that's not something the language can fix. I really hope they don't change things too much.
Re:Python 3.11 vs 3.10 performance by joshmaker on Wednesday July 06, 2022 @03:42PM (#32005829)
The new error messages are much better. Take a look at this example:

  Traceback (most recent call last):
    File "calculation.py", line 54, in <module>
      result = (x / y / z) * (a / b / c)
                ~~~~~~^~~
  ZeroDivisionError: division by zero

In this new version it's now obvious which variable is causing the 'division by zero' error.

https://docs.python.org/3.11/whatsnew/3.11.html#enhanced-err...

Re:Python 3.11 vs 3.10 performance by bredren on Wednesday July 06, 2022 @06:27PM (#32007660)
For a lot of folks dealing with getting something working, its way more useful to have the most likely spot where the error is occurring spit out for them like this.

Stack trace is useful, especially in understanding how the code is working in the system.

But if the goal is solve the problem with the code you've been working on, existing traces are way too verbose and if anything add noise or distract from getting to productive again.

I could see tracebacks get swifty terminal UI that shows only the pinpointed error point that can be accordion'd out to show the rest.

Re:Python 3.11 vs 3.10 performance by dx034 on Thursday July 07, 2022 @02:38AM (#32010637)
You've clearly been working with the right people. I constantly get screenshots of error messages (not even the text of the traceback so that I could copy it) with questions about what that means. It takes some training for people to be able to read tracebacks correctly, especially if they have no other programming experience.
Re:Python 3.11 vs 3.10 performance by e-master on Wednesday July 06, 2022 @06:32PM (#32007723)
I only rarely delve into the python world, but as a .net developer I always found it odd and confusing that the error message comes after the stack trace in python. I don’t see people complaining about it, so maybe it’s just a matter of habit?
Re:Python 3.11 vs 3.10 performance by kzrdude on Wednesday July 06, 2022 @07:23PM (#32008133)
I guess it's just a habit even if I think it's mildly confusing too.

But: it's kind of a virtue that the most important error message comes last. Far too many (ahead of time) compilers output too many errors. The most likely error to matter is the first one, but it is often scrolled way off screen; the following errors might even just be side effects that mislead or confuse the new users.

Re:Python 3.11 vs 3.10 performance by AdamN on Thursday July 07, 2022 @09:12AM (#32013646)
On the CLI, you'll see the error just above the prompt and then the stack trace above that. It makes sense since the user is always reading up the terminal in reverse order.
Re:Python 3.11 vs 3.10 performance by actuallyalys on Wednesday July 06, 2022 @05:07PM (#32006846)
I don’t think they’re that bad, but I think the bar has (fortunately) become higher.
Re:Python 3.11 vs 3.10 performance by urthor on Wednesday July 06, 2022 @11:07PM (#32009513)
Exactly right.

The old Python stack trace, circa 2015, was the best in the world.

The new ones are even better.

The gap between Python and Rust, and the JVM languages/C/C++, is increasingly widening.

Stack trace is one of the areas Python and Rust are making the case for why they're the languages of the future.

Re:Python 3.11 vs 3.10 performance by kirillbobyrev on Thursday July 07, 2022 @03:09AM (#32010809)
Not saying that there isn't room for improvement but I wouldn't call the Python error messages "cryptic". C++, on the other hand...
Re:Python 3.11 vs 3.10 performance by gpderetta on Thursday July 07, 2022 @06:06AM (#32011976)
I program in both C++ and Python every day and I'm fluent in both. I would prefer 10000 lines template instantiation backtraces every time to the average python error message.

Clang and GCC error messages have come a loooong way in the last 10 years or so and their quality is impressive.

To be fair, I'm currently stuck at python 3.6 ATM and I hear that python has also improved a lot since.

Re:Python 3.11 vs 3.10 performance by bilsbie on Wednesday July 06, 2022 @11:47AM (#32002675)
When is that coming? Yes the error messages are confusing in a big list comprehension.
Re:Python 3.11 vs 3.10 performance by alexfromapex on Wednesday July 06, 2022 @12:17PM (#32003114)
Re:Python 3.11 vs 3.10 performance by teekert on Wednesday July 06, 2022 @01:15PM (#32003934)
Whenever I nest a list comprehension I feel dirty anyway, I wonder if a better error message will help solve something that is just hard to grasp sometimes.
Re:Python 3.11 vs 3.10 performance by 8organicbits on Wednesday July 06, 2022 @01:47PM (#32004376)
"Debugging is twice as hard as writing the code in the first place. Therfore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Rajanand

Although, personally, I enjoy python list comprehensions.

Re:Python 3.11 vs 3.10 performance by timoteostewart on Wednesday July 06, 2022 @01:55PM (#32004479)
I believe that quote about debugging being twice as hard as coding is by Brian Kernighan.

https://en.m.wikiquote.org/wiki/Brian_Kernighan

Re:Python 3.11 vs 3.10 performance by 8organicbits on Wednesday July 06, 2022 @02:21PM (#32004774)
Maybe, I grabbed the first matching quote I found. I can't attribute it on my own.
Re:Python 3.11 vs 3.10 performance by timoteostewart on Wednesday July 06, 2022 @03:00PM (#32005302)
No sweat. I’m a quote nerd so it jumped out at me
Re:Python 3.11 vs 3.10 performance by mikepurvis on Wednesday July 06, 2022 @02:14PM (#32004711)
Not even with nesting— as soon as it's long enough that I'm breaking it into multiple lines, I'm immediately like, okay this is long enough to just be a normal for-loop, or maybe a generator function. But it doesn't need to be a list comprehension any more.
Re:Python 3.11 vs 3.10 performance by polski-g on Wednesday July 06, 2022 @01:27PM (#32004088)
All list comprehensions should error with message "FATAL PARSE ERROR: Will you really understand this in six months?"
Re:Python 3.11 vs 3.10 performance by teekert on Thursday July 07, 2022 @04:52AM (#32011486)
Something like this I find ok (just a filter):

    [x for x in y if x is not z]
Sometimes though, facilitated by Jupyter's notebooks making executing as you build the code very easy, I create a beast like this:

    [os.path.join([x for x in y if x is not z]) + '/bin' for a in b if 'temp' not in a]
Yes this is fictional and probably contains an error but you get the point, you can filter lists of lists like this, but it's really unfriendly to anyone trying to understand it later.
Re:Python 3.11 vs 3.10 performance by wheelerof4te on Wednesday July 06, 2022 @03:11PM (#32005422)
List comprehensions should be used for making quick lists that contain simple calculations or filters.

Anything more will come back to bite you later.

Re:Python 3.11 vs 3.10 performance by TremendousJudge on Wednesday July 06, 2022 @01:42PM (#32004290)
If people complain about Rust's borrow checker already...
Re:Python 3.11 vs 3.10 performance by psalminen on Wednesday July 06, 2022 @01:49PM (#32004398)
Big list comprehensions cause confusion in many ways. KISS is crucial with them.
Re:Python 3.11 vs 3.10 performance by mcronce on Wednesday July 06, 2022 @02:08PM (#32004647)
I agree. The rule of thumb I follow is that if the list comp doesn't fit on one line, whatever I'm doing is probably too complicated for a list comp.
Re:Python 3.11 vs 3.10 performance by gpderetta on Thursday July 07, 2022 @06:14AM (#32012058)
But why? The only reason is that error messages are awful and intermediate values can't be named. There is in principle no reason why list comprehension need to be worse than straight line for loops. The deciding factor to chose betwee the two should be whether I'm writing something for side effects as opposed to for its value.
Re:Python 3.11 vs 3.10 performance by mcronce on Saturday July 09, 2022 @02:49AM (#32032530)
You just listed two very good reasons: Intermediate values can't be named, and error messages are awful.
Re:Python 3.11 vs 3.10 performance by agumonkey on Wednesday July 06, 2022 @01:29PM (#32004114)
In a way it's a nice incentive not to nest too much (I have that tendency too)
Re:Python 3.11 vs 3.10 performance by yantrams on Wednesday July 06, 2022 @02:04PM (#32004604)
Re:Python 3.11 vs 3.10 performance by ChadNauseam on Wednesday July 06, 2022 @02:38PM (#32005033)
Hold mine :D https://github.com/anchpop/genomics_viz/blob/master/genomics...

That's one expression because it used to be part of a giant comprehension, but I moved it into a function for a bit more readability. I'm considering moving it back just for kicks though.

My philosophy is: if you're only barely smart enough to code it, you aren't smart enough to debug it. Therefore, you should code at your limit, to force yourself to get smarter while debugging

Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @04:10PM (#32006163)
Yours is nice and readable. Parents' one is not, but it feels like the indentation is deliberately confusing. I'd lay it out like so:

   tags = list(set([
     nel
     for subli in [
       mel
       for subl in [
         [[jel.split('/')[2:] for jel in el] 
         for el in classified
       ]
       for mel in subl 
     ]
     for nel in subli
     if nel
   ]))
Still not very readable, tho. But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that doesn't scale at all.

Side note: one other thing I always hated about those things in Python is that there's no way to bind an intermediary computation to a variable. In C# LINQ, you can do things like:

   from x in xs
   let y = x.ComputeSomething()
   where y.IsFoo && y.IsBar
   select y
In Python, you have to either invoke ComputeSomething twice, or hack "for" to work like "let" by wrapping the bound value in a container:

   y
   for x in xs
   for y in [x.ComputeSomething()]
   if y.IsFoo and y.IsBar
It's not just about not repeating yourself or not running the same code twice, either - named variables are themselves a form of self-documenting code, and a sequence pipeline using them even where they aren't strictly needed can be much more readable.
Re:Python 3.11 vs 3.10 performance by dragonwriter on Wednesday July 06, 2022 @10:20PM (#32009272)
> But that's largely due to Python's outputs-first sequence comprehension syntax being a mess that doesn't scale at all.

Choices like wrapping a set constructor around a list comprehension rather than just using a set comprehension don't help, neither does using multiple nested no-transformation comprehensions laid out that way.

By hand, on mobile, I think this reduces to (assuming “from itertools import chain” in the file header):

   tags = list({
     nel
     for subli in chain.from_iterables(chain(
       [[jel.split('/')[2:] for jel in el] 
       for el in classified))
     for nel in subli
     if nel
   })
Re:Python 3.11 vs 3.10 performance by andrewaylett on Thursday July 07, 2022 @10:56AM (#32014904)
Since it's all generator expressions internally anyway, an intermediate doesn't actually add appreciable (any?) runtime overhead but does make the code more readable:

    ys = (x.ComputeSomething() for x in xs)
    result = [y for y in ys if y.isFoo and y.isBar]
Re:Python 3.11 vs 3.10 performance by texaslonghorn5 on Wednesday July 06, 2022 @04:15PM (#32006225)
Does the new walrus := in python solve your problem?
Re:Python 3.11 vs 3.10 performance by roelschroeven on Wednesday July 06, 2022 @06:29PM (#32007685)
The official name is assignment expression, which can be good to know to find documentation. Here's the PEP: https://peps.python.org/pep-0572/
Re:Python 3.11 vs 3.10 performance by otherme123 on Thursday July 07, 2022 @01:57AM (#32010407)
Curiously, searching the docs for "walrus" (https://docs.python.org/3/search.html?q=walrus) yields exactly the results you need, but searching for "assignment expression" yields the previous results mixed a lot of other results due to the word "expression".
Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @09:25PM (#32008944)
Not really; where would you put it inside the []?
Re:Python 3.11 vs 3.10 performance by dragonwriter on Wednesday July 06, 2022 @09:33PM (#32008981)
In the example from the end of the earlier post:

   y
   for x in xs
   if (y:=x.ComputeSomething()).IsFoo and y.IsBar
Re:Python 3.11 vs 3.10 performance by int_19h on Wednesday July 06, 2022 @11:20PM (#32009574)
I suppose that works, although readability is not great, to put it mildly.
Re:Python 3.11 vs 3.10 performance by texaslonghorn5 on Wednesday July 06, 2022 @11:21PM (#32009580)
Much better would be

[(y:=x.ComputeSomething()) for x in xs if y.IsFoo and y.IsBar]

Re:Python 3.11 vs 3.10 performance by int_19h on Thursday July 07, 2022 @03:01AM (#32010755)
That doesn't work, because that first subexpression is evaluated last, after "if".
Re:Python 3.11 vs 3.10 performance by ChadNauseam on Wednesday July 06, 2022 @11:37PM (#32009649)
For another example of how to use the walrus operator, I use it multiple times in the code I linked a few comments up
Re:Python 3.11 vs 3.10 performance by teddyh on Thursday July 07, 2022 @07:52AM (#32012807)
Re:Python 3.11 vs 3.10 performance by agumonkey on Wednesday July 06, 2022 @04:02PM (#32006055)
That code is fine in my book, there's more static code than for-comp and it's at work 3 level deep with no hard coupling between levels. Also well spaced and worded.
Re:Python 3.11 vs 3.10 performance by wvh on Wednesday July 06, 2022 @03:12PM (#32005438)
I don't think this is bad code. Maybe not the most Pythonic in the imperative sense, but certainly you could come across such constructs in languages that are more oriented towards functional programming. This could also be solved in a more readable manner by having a generator pipeline, though it would be a good idea to see what the performance of chained generators is like in your version and flavour of Python.
Re:Python 3.11 vs 3.10 performance by kzrdude on Wednesday July 06, 2022 @07:26PM (#32008147)
It's “not even” using for nesting in list comprehensions. This kind of thing:

   [x for stop in range(5) for x in range(stop)]