Ya, this is because the initialized object is stored as the default parameter, and is not initialized every time the function runs.
def functy(dicty={}):
print('dicty=', dicty)
return dicty
output = functy()
# dicty = {}
output.update({'key': 'value'})
functy()
# dicty = {'key': 'value'}I think this behavior is actually "default args" but reversed. In case of default args it is the mutability that causes the nasty surprise (ooops, all calls share the same object, which is mutable). In case of the "+=" it is the _immutability_ that causes the surprise - I would not expect "obj.a += ..." to create an instance attribute that shadows the class one.
I think the practical conclusion here is "don't call += on objects that don't support it - because it may work and this is not what you want!)".
I'm curious if you see it the same way?
It seems to, but, actually, it doesn't.
> but wtf'ingly also alters the value of the class attribute "A.b" which completely alters the behavior of A.
Actually, all it does is call the __iadd__ method on the list that it is the value of A.b (which modifies that list in-place.)
The value of A.b can, incidentally be accessed via obj.b because obj.b does not exist and the lookup path for instance attributes includes the class.
There are two things going on here there are potential sources of confusion, because they behavior differently in different circumstances: object member access (which can get members from the object or, if they don't exist on the object, from the class) and the += operator, which calls the __iadd__ method on the left value or acts as add and then assign to the name on the left, if that name doesn't reference a value that supports __iadd__.
That is not correct. += will always perform the assignment, hence the classic oddball concatenation:
>>> v = ([],)
>>> v
([],)
>>> v[0] += [1]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> v
([1],)
l = obj.b
l.extend(2)
obj.b = l
>>> class A: b = [1]
...
>>> a = A()
>>> a.b += (2,)
>>> vars(a), a.__dict__
({'b': [1, 2]}, {'b': [1, 2]})
>>> A.b = [5]
>>> a.b
[1, 2]Side note: the object can be immutable, but then the (variable or attribute) left side should be replaced with a new immutable object.
I thought of this as though the list A.b was marked static - then any modification of b through any instance of A modifies the shared static member
>>> class A:
... b = [1,]
...
>>> obj = A()
>>> obj.b.append(2)
>>> A.b
[1, 2]
>>> foo = A()
>>> foo.b
[1, 2]
>>>But what does that have to do with assignment?
>>> a = 3
>>> a += 4
>>> print(a)
7
Obviously, this is inspired by the C += operator, and behaves accordingly, when it's not being weird.
The difference is in how the Class attributes are affected i.e. the values of "A.a" and "A.b".
# obj.a += (2,) has no impact on A.a since tuples
# are immutable and copies are made.
# This is unexpected from a C/C++ intuition.
>>> print(A.a)
(1,)
# obj.b += [2,] has an indirect impact on A.b
# since lists are mutable and updates are made in place.
# This is unexpected for beginners in Python since it
# deviates from how int, str and tuples behave as "+="
# can be used for int, str, tuples and lists.
>>> print(A.b)
[1, 2]
No, they don't.
obj.b does not exist, and no assignment to it takes place, and there is no “indirect effect”.
A.b exists, and because class attributes can be accessed as if they were members on instances, and because A.b has an implementation of in-place addition, that implementation is called on A.b instead of an assignment to obj.b. That is the only thing that happens as a result of the statement that looks like it might be an assignment to obj.b. The effect on A.b is the only action, not an indirect effect of an assignment.
It necessarily alters the value of A.b, since `+=` is specifically overridden to update the list in-place. With lists, `a += b` is specifically not an alias for `a = a + b`, instead its behaviour is closer to
a.extend(b)
a = a
Note to readers unfamiliar with Python: class attributes is a special thing, different than instance attributes.
Definitely one of the trickier things with Python for sure.
async function totalSize(fol) {
const files = await fol.getFiles();
let totalSize = 0;
await Promise.all(files.map(async file => {
totalSize += await file.getSize();
}));
// totalSize is now way too small
return totalSize;
}
I guess the problem in python is different though, it's about having some data structures that are immutable (like tuple) and others that are mutable (like list)
To fix this, simply await the result of getSize first, and then add the result to totalSize using +=
That is the sole source of the issue, and exactly what the comment you're replying to talks about.
The problem is that `a += b` desugars to `a = a + b`, so if `b` is an await you get
totalSize = totalSize + await file.getSize();
That must have been a fun one to track down!
class A:
a = (1,)
b = [1,]
def __init__(self):
self.a = (1,)
self.b = [1,]
A.a # this is the class attribute
(1,)
A.b # class attribute
[1]
obj = A()
obj.a # instance attribute
(1,)
obj.b
[1]
obj.a += (2,)
obj.b += [2,]
A.a
(1,) # still the same class attribute
A.b
[1]
obj.a
(1, 2) # instance attribute appended
obj.b
[1, 2]If I had had magic wand, I'd make operations on class attributes from an instance a syntax error, and only allow it from the class name, and perhaps with special syntax like in C++, e.g.:
A::a = (1,)
A::b = [1,]
> # tuples are immutable and don't have "+="
What do you mean? Because it looks to me tuples have "+=":
a = (1,)
a += (3,)
# a = (1, 3)
Python is more of a lisp. The assignment semantics are different to C. Actually, you're letting the name "a" point to the newly created object.
In C, a variable is a memory position you can fill and point to.
In Python, a variable is a name that points to a memory position.
Similarity to C is no accident here, since the lexical scoping concepts in Lisps like Scheme and CL, as well as in C, both trace back to Algol.
Classic Lisp global/dynamic variables may store a global value a "value cell" which is closely tied to the symbol itself (perhaps stored in it).
And in that sense, C assignment and Lisp assignment are different beasts. Mainly in that - again in the semantics of the high-level description - "memory reservation" in C happens when we declare the left-hand side of an assignment and in Lisp it happens when we construct the right-hand side.
Though if you can show me a mainstream Lisp where the assignment works like setting a memory cell and not like a bind, I'm happy to be proven wrong.
The right hand side reserves memory only in the sense that some heap object is allocated (if that is the case), but that is secondary to the assignment; and that is the same as C also, as in:
f = fopen(...); // right hand allocates stream; pointer moves into variable.
The mainstream Lisps clearly separate binding from assignment.
(let (x) ;; x refers to a freshly allocated cell (nil-initialized, in Common Lisp)
...
(setq x 42) ;; cell is clobbered, replacing nil with 42.
...)
1. A variable is a "binding in the variable namespace" (Glossary)
2. A binding is an association between a name and a value (Glossary)
3. Yet, the description of SETQ uses language like: "First form1 is evaluated and the result is stored in the variable var1".
4. LET is described like this "let and let* create new variable bindings and execute a series of forms that use these bindings". No binding creation semantics is mentioned for SETQ.
Results can only be stored in storage places; nothing can be stored in an "association between a name and a value", unless that association is actually set up through a memory cell: the name refers to a location and the location holds a value. SETQ isn't described in terms of breaking an old binding and setting up a new one..
In the case of dynamic and global variables, there is a term "value cell" which the Glossary defines: like this: "The place which holds the value, if any, of the dynamic variable named by that symbol, and which is accessed by symbol-value. "
Let's turn our attention to Scheme. R7RS says in 3.1. Variables, syntactic keywords, and regions this:
"An identifier can name either a type of syntax or a location where a value can be stored."
and:
"An identifier that names a location is called a variable and is said to be bound to that location."
"The value stored in the location to which a variable is bound is called the variable’s value."
The next sentence is a kicker, and can be regarded as a criticism of the ANSI CL definition of binding:
"By abuse of terminology, the variable is sometimes said to name the value or to be bound to the value."
If you think that Lisp variables are bindings to values, then according to the Scheme maintainers, you've fallen victim to abuse of terminology. :)
Thanks!
But if l does support __iadd__, then l += r becomes l.__iadd__(r).
This behavior for list and tuple is familiar from regular local variables. Together with the fact that obj.x = ... always means "create or rebind an instance variable" never "rebind the A.x class variable", I think the example is less surprising.
>>> print(obj.a, obj.b)
(1, 2) [1, 2]