WTFPython: Exploring and understanding Python through surprising snippets

Re:WTFPython: Exploring and understanding Python through sur... by ghusbands on Tuesday May 31, 2022 @07:35AM (#31568350)

There's a similar surprising += 'problem' in JS. If you have the following code:

  async function totalSize(fol) {
    const files = await fol.getFiles();
    let totalSize = 0;
    await Promise.all(files.map(async file => {
      totalSize += await file.getSize();
    }));
    // totalSize is now way too small
    return totalSize;
  }

You get an overly low totalSize. It's caused by 'a += b' expanding to 'a = a + b', and the double-mention of 'a' creating a concurrency issue. If '+=' were a single operation with the right-hand-side being calculated first, it wouldn't be an issue.

[ Parent | on HN ]

Re:WTFPython: Exploring and understanding Python through sur... by sharno on Tuesday May 31, 2022 @02:36PM (#31573136)

I don't think this is a concurrency problem. It's a closure capture problem.

I guess the problem in python is different though, it's about having some data structures that are immutable (like tuple) and others that are mutable (like list)

[ Parent | on HN ]

Re:WTFPython: Exploring and understanding Python through sur... by DrammBA on Tuesday May 31, 2022 @01:37PM (#31572405)

The problem here is the closure on totalSize and where you're awaiting, totalSize is going to be 0 for all files and then you await for the result of getSize, so in the end totalSize is going to be equal to the size of the last file that completes the getSize call.

To fix this, simply await the result of getSize first, and then add the result to totalSize using +=

[ Parent | on HN ]

Re:WTFPython: Exploring and understanding Python through sur... by masklinn on Tuesday May 31, 2022 @03:11PM (#31573584)

> and where you're awaiting

That is the sole source of the issue, and exactly what the comment you're replying to talks about.

The problem is that `a += b` desugars to `a = a + b`, so if `b` is an await you get

    totalSize = totalSize + await file.getSize();

Since javascript evaluates left to right, it first evaluates `totalSize`, gets zero, then `file.getSize()`, then suspends waiting for the result... at which point the handler for the next file can run, doing the exact same thing, repeat for all files.

[ Parent | on HN ]

Re:WTFPython: Exploring and understanding Python through sur... by aastronaut on Tuesday May 31, 2022 @12:11PM (#31571287)

stupid question incomming: isn't that a '+= await' problem?

[ Parent | on HN ]

Re:WTFPython: Exploring and understanding Python through sur... by tln on Tuesday May 31, 2022 @11:54AM (#31571038)

Wow, that is surprising.

That must have been a fun one to track down!

[ Parent | on HN ]