WTFPython – Understanding Python through surprising snippets

Re:WTFPython – Understanding Python through surprising snippets by masklinn on Monday December 23, 2019 @08:18AM (#21863397)

It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change.

    for(var i=0; i<10; ++i) {
        powersOfX.push(x => x^i);
    }

will behave the same way because it does the same thing, so will e.g.

    for i := 0; i<10; i++ {
        powers_of_x = append(powers_of_x, func(x int) int { return x ^ i })
    }

in Go.

One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible).

The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over.

Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions.

In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas:

    for i in range(10):
        powers_of_x.append(lambda x, i=i: x^i)

[0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings)

[1] and remains a regular issue in async code fighting over closed-over context

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by aidos on Monday December 23, 2019 @09:12AM (#21863674)

You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by gpderetta on Monday December 23, 2019 @08:59AM (#21863604)

C++:

   std::vector<std::function<int(int)>> powers_of_x;
   for (int i = 0; i < 10; ++1)
       power_of_x.push_back([=](int x) { return std::pow(x, i); });

works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by masklinn on Monday December 23, 2019 @11:05AM (#21864525)

…

> excluding the special case of low-level languages with capture clauses

Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by gpderetta on Monday December 23, 2019 @11:52AM (#21864908)

To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid.

Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by loopz on Monday December 23, 2019 @09:49AM (#21863897)

Works intuitively in golang:

https://play.golang.org/p/pn2jBTaTNS8

  package main
  
  import (
   "fmt"
  )
  
  // return a^n
  func Power(a, n int) int {
   var i, result int
   result = 1
   for i = 0; i < n; i++ {
    result *= a
   }
   return result
  }
  
  func main() {
   fmt.Println("Hello, playground")
   x := 2
   var powers_of_x []int
   for i := 0; i < 10; i++ {
    powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x))
   }
   fmt.Println(powers_of_x)
  }

  //Output:
  //Hello, playground
  //[1 2 4 8 16 32 64 128 256 512]

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by gpderetta on Monday December 23, 2019 @10:38AM (#21864252)

I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).

[ Parent | on HN ]

Re:WTFPython – Understanding Python through surprising snippets by masklinn on Monday December 23, 2019 @11:04AM (#21864505)

I provided a Go example demonstrating that it does not. Your version

1. uses a slice of integers which completely misses the point

2. uses an IIFE which completely misses the point

The point is to look at what happens when you close over the iteration variable.

[ Parent | on HN ]