News for hackers. Stuff that matters.
Re:Python quirks by emef on Friday July 12, 2013 @12:39PM (#6033546)
I'm surprised using mutable value for a default in a keyword argument wasn't mentioned. This certainly tripped me up early in learning python:

    def fn(x, my_dict={}):
        my_dict[x] = x * 2
        return my_dict

    >>> fn(1)
    {1: 2}
    >>> fn(2)
    {1: 2, 2: 4}
(I would have expected the second call to return {2: 4} when I was learning python)
Re:Python quirks by jliechti1 on Friday July 12, 2013 @01:48PM (#6033931)
This was discussed pretty recently in this thread on "Python Newbie Mistakes, Part 1".

Default values for functions in Python are instantiated when the function is defined, not when it’s called.

Thread: https://news.ycombinator.com/item?id=5999772

Direct link: http://blog.amir.rachum.com/post/54770419679/python-common-n...

It looks like the author posted a "part 2" to HN as well, but it never made the front page.

http://blog.amir.rachum.com/post/55024295793/python-common-n...

Re:Python quirks by kyllo on Friday July 12, 2013 @01:48PM (#6033933)
This is pretty bizarre, and seems like it would cause a memory leak if you don't know what you're actually doing. You're allocating a my_dict object on the heap, in a field of the function object, so my_dict never goes out of scope and never gets garbage collected until the function fn itself does, right?

Whereas if you did this instead:

    def fn(x):
        my_dict={}
        my_dict[x] = x * 2
        return my_dict
You'd get what you'd expect--a new my_dict object gets created, returned, and then goes out of scope every time fn is called, so it would get garbage collected once there are no more references to that return value. (I think...)

(I don't know that much about how memory allocation and GC works in Python yet, just trying to learn!)

Re:Python quirks by baq on Friday July 12, 2013 @04:02PM (#6034844)
this is not at all bizarre, you just need to know what dynamic means. the best answer i could come up with is "definition is execution". for an enlightening moment, see this piece of code:

    def a():
        print "a called"
        return []

    def fn(x=a()):
        x.append(1)
        print x

    fn()
    fn()
    fn()
i suggest typing this directly into the interpreter instead of a script for better effect.
Re:Python quirks by akkartik on Friday July 12, 2013 @10:39PM (#6036472)
You got me thinking about how python differs from lisp in this respect, and the subtle way that a functional API has helped this specific situation.

While lisp suffers from the same "definition is execution" gotcha, the effects are far rarer in practice because () is immutable and interned while [] is mutable and usually generated afresh each time it's executed.

  $ python
  >>> [] is []
  False

  $ sbcl
  * (eq () ())
  t
Since [] is mutable, appends of [] can do superficially 'the right thing'.

  >>> a = []
  >>> a.append(4)
  >>> a
  [4]

  * (setq a ())
  * (nconc a '(34))
  (34)
  * a
  ()
But there's a reason lisp does seemingly the wrong thing. According to the spec, nconc skips empty arguments (http://www.lispworks.com/documentation/HyperSpec/Body/f_ncon...) Reading between the lines, I'm assuming this makes sense from a perspective in lisp where we communicate even with 'destructive' operations through their return value. This is more apparent when you consider nreverse:

  * (setq a '(1 2 3 4))
  * (nreverse a)
  (4 3 2 1)
  * a
  (1)
Destructive operations can reuse their input, but they're not required to maintain bindings. There is no precise equivalent of python's .reverse(). Instead, a common idiom is:

  * (setq a (nreverse a))
It seems like a weird design decision, but one upshot of it besides encouraging a more functional style is that this optional-arg gotcha loses a lot of its power in lisps. It's very rare to define a default param of a non-empty list, and empty lists can't be modified without assigning to them.
Re:Python quirks by bfs on Friday July 12, 2013 @05:52PM (#6035459)
Great explanation!
Re:Python quirks by jvdongen on Friday July 12, 2013 @01:55PM (#6033966)
Basically. The idiomatic solution for this in python is:

  def fn(x, my_dict=None):
     if my_dict is None:
         my_dict = {}
      ...
Re:Python quirks by brownbat on Friday July 12, 2013 @10:15PM (#6036431)
You can use a sentinel if you want to allow None as an argument. More on that, along with a way to use mutable defaults productively:

http://effbot.org/zone/default-values.htm

Re:Python quirks by rbanffy on Saturday July 13, 2013 @04:17AM (#6037028)
I never did it myself, but this behavior is very useful if you want something to be used across invocations, like a cache.

To limit its size, you can use the ideas on http://stackoverflow.com/questions/2437617/limiting-the-size...

Re:Python quirks by rbanffy on Friday July 12, 2013 @03:46PM (#6034747)
In other words, defaults are static (an in C) and initialized when the function is defined, not when it's called.
Re:Python quirks by adamtj on Friday July 12, 2013 @10:40PM (#6036477)
It is perhaps more correct to say "when the function is instantiated". Functions in python aren't exactly "defined". Functions are objects same as everything else. The "def" keyword is how you call the constructor for a function object. The kwarg default values are evaluated once at object construction time and saved.

The "def" statement may construct different distinct function objects from a single definition. Each distinct function object has different default value objects, but each function object reuses its own default value objects each time it's called.

Consider this:

    def outer_fn():
        def inner_fn(foo={}):
            # The id() function returns an internal
            # object identifier.  Different objects 
            # have different ids.
            print id(foo)
        return inner_fn
    
    inner_fn_1 = outer_fn()
    inner_fn_2 = outer_fn()
    inner_fn_1()
    inner_fn_1()
    inner_fn_2()
    inner_fn_2()
Re:Python quirks by rbanffy on Saturday July 13, 2013 @04:13AM (#6037021)
You are absolutely correct, of course. I was using a C-ism.
Re:Python quirks by bsimpson on Friday July 12, 2013 @01:03PM (#6033688)
So it's instantiating the dict in the declaration and recycling it between invocations? I wouldn't have expected that either.
Re:Python quirks by bskap on Friday July 12, 2013 @02:04PM (#6034030)
"def" in Python is an executable statement, not a declaration- it creates a function object that gets bound to the specified name in the local namespace. The defaults are evaluated when the function is created.
Re:Python quirks by chrisdotcode on Friday July 12, 2013 @01:16PM (#6033759)
Rather the dict is bound during "compile" time, and not at each individual invocation (which would be much better, IMO). And during each invocation, it doesn't get re-bound.
Re:Python quirks by baq on Friday July 12, 2013 @04:04PM (#6034857)
it can't be, that wouldn't make sense. "def" is no different than "for" or "if".
Re:Python quirks by js2 on Friday July 12, 2013 @04:06PM (#6034876)
It's useful for memoization though and is where I've typically used it.