Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @07:37AM
(#7005140)
And here's a more magical version that wraps it with a decorator. If you're gonna do this I kind of like this way better, but I'm pretty sure your average python programmer finds it obscene:
from pprint import pprint
def default_init(init_func):
arg_count = init_func.func_code.co_argcount
arg_vars = init_func.func_code.co_varnames[1:1+arg_count]
def wrap_init(self, *args, **kwargs):
idx = 0
for arg in arg_vars:
if arg in kwargs:
setattr(self, arg, kwargs[arg])
else:
setattr(self, arg, args[idx])
idx += 1
init_func(self, *args, **kwargs)
return wrap_init
class testing(object):
@default_init
def __init__(self, a, b, c):
pass
x = testing(1, 2, 3)
pprint(x.a)
pprint(x.b)
pprint(x.c)
y = testing(1, 2, c=3)
pprint(y.a)
pprint(y.b)
pprint(y.c)
[note: I'm aware this does not do even remotely a complete job as a decorator. Use at your own risk, it is a demonstration only.]
Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @08:20AM
(#7005258)
One thing I like a lot better about it in particular is that it doesn't reach into the stack frame, which strikes me as a very messy grab at internals people should never ever touch. The arity and name information from the compiled function object, however, has legitimate use in tooling and I suspect a lot of decorators do this sort of thing.
It's an interesting balance between one being more magical in its use of the system (inspecting live stack frames) and the other being more magical in appearance (having a function with just pass in it actually do something).
Re:Making Python's __init__ method magical
by n0on3 on Friday January 03, 2014 @09:30AM
(#7005542)
Here is another version implemented as a decorator.
Actually I’d also like not to use inspect, but didn’t find any nice way to get the kwargs default values without it, any suggestion?
import inspect
def autoinit(decorated_init):
def _wrap(*args, **kwargs):
obj, nargs= args[0], args[1:]
names = decorated_init.func_code.co_varnames[1:len(nargs)+1]
nargs = {k: nargs[names.index(k)] for k in names}
kwa_keys = decorated_init.func_code.co_varnames[len(nargs)+1:]
kwa_defaults = inspect.getargspec(decorated_init).defaults
for k in kwa_keys:
nargs[k] = kwargs[k] if (k in kwargs) else kwa_defaults[kwa_keys.index(k)]
for k, v in nargs.iteritems():
setattr(obj, k, v)
return decorated_init(*args, **kwargs)
return _wrap
if __name__ == '__main__':
class Foo():
@autoinit
def __init__(self, a, b, x=0, y=0):
self.p = 'Hello, I am set inside the __init__'
self.s = self.a + self.b + self.x + self.y
f = Foo(1, 2, x=120, y=5)
for param in 'abxyps':
print param, '=>', getattr(f, param)
f = Foo(31, 33, x=64)
for param in 'abxyps':
print param, '=>', getattr(f, param)
f = Foo(7, 121)
for param in 'abxyps':
print param, '=>', getattr(f, param)
Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @07:19AM
(#7005066)
Really really minor nitpick, but when talking about languages and their traits these little details can become important for the reader to understand, lest it confuse them about other languages:
> While many people think of __init__ as a constructor, that’s not really the case. Rather, __init__ is invoked on our new object after it has been created
I'm not aware of a language (though I don't claim to know them all!) in which this is not exactly the definition of a constructor. In C++, `operator new` does the allocation, the constructor is called after. In java it's similar. In ruby it's Class#alloc (which is called by Class#new as roughly `ClassName.alloc.tap{|o| o.initialize(*args); o}`)
So the reader is correct to interpret it as a constructor. Constructors initialize the state of an uninitialized object.
Re:Making Python's __init__ method magical
by Genmutant on Friday January 03, 2014 @07:34AM
(#7005129)
In Python you have a "real" constructor you can use: __new__. So I think it's good not to call __init__ a constructor. Unless of course you differ in my opinion that __new__ is a constructor?
Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @07:38AM
(#7005145)
I do. __new__ is effectively an allocator, or an interface to it. It allocates an unconstructed object. Then the constructor, __init__ is called on that.
Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem from that. I do think the distinction is meaningful and important in languages that separate these two things out, though.
Re:Making Python's __init__ method magical
by reuven on Friday January 03, 2014 @07:48AM
(#7005167)
I'll have to check my terminology to understand this better. But I can assure you that most of the programmers who take my Python classes (who tend to come from Java, C#, and C++) are rather surprised that the constructor modifies an existing object, and doesn't actually create the object.
That said, I'm totally willing to believe that neither they nor I understood the difference between an "allocator" and a "constructor." I'll check into this further, both for myself and for the sake of future writing and lectures!
Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @08:13AM
(#7005239)
Well, they have a poor understanding[1] of the sequence of events in C++ if they think that the constructor allocates the object there (I won't speak too strongly to Java or C#, since I don't work in them if I don't have to, but I believe that they both work in a very similar way). This would in fact be impossible, since many constructors are usually called for any given object (and all constructors in the object's graph are always called, unlike the more traditional OO languages where the upcall is usually explicit). Also because where an object is allocated is a decision of the caller. It can be allocated with new on the heap, on the stack with a simple declaration, or at an arbitrary memory location with placement new.
In C++ the heap object initialization goes like this:
Obj *obj = new Obj;
This calls either `Obj::operator new` or the globally scoped `::operator new`, which somehow obtains an area of memory at least sizeof(obj) bytes long and returns it. After that calls to the object's constructors are generated and called.
`operator new` is an allocator and the constructor initializes the raw bytes returned from it.
You can actually do the process explicitly in its entirety like so using placement new, which is a special form that calls the constructors on an arbitrary location:
You can't call constructors directly (except with a C++0x feature that lets you call them from a same-level constructor) because of the issue of how they're ordered based on the object graph. They have a precondition of their parent class having been fully constructed.
[1] Note: I don't really mean this as a judgement, understanding of C++ internals are pretty poor in a lot of programmers and for pretty good reasons a lot of the time.
Re:Making Python's __init__ method magical
by usea on Friday January 03, 2014 @11:45AM
(#7006366)
Hi, I'm a C# developer who has done some Java and C++ in the past. You may consider me in the box of people who thinks the constructor creates the object, and the object not existing until the constructor finishes. The allocation of memory is not what I would count as creation of an object, as a model of memory is basically unrelated to the idea of objects and language semantics as I use them. If the object cannot be used, I would consider it not yet created.
I am not trying to argue (on the contrary, I would defer to your expertise), but rather I mean to add a small data point of what perspective developers with my background may have.
Re:Making Python's __init__ method magical
by andrewflnr on Friday January 03, 2014 @02:13PM
(#7007405)
In the discussion of C++ constructors, someone usually (IME) says something like, "now remember, it's not a real Foo until the constructor finishes!" because the invariants haven't been set up yet. That could be where the misunderstanding you mentioned comes from.
Re:Making Python's __init__ method magical
by radiowave on Friday January 03, 2014 @07:55AM
(#7005186)
In Smalltalk, 'new' is a class method which allocates a new instance, but which then has no special access to the internals of that newly created instance. An instance method, 'initialize' is then almost always used to set the initial state (by convention - this is not an actual language feature). Most commonly, initialize is called by new, but this again is just a convention, and a fairly recent one.
Re:Making Python's __init__ method magical
by stormbrew on Friday January 03, 2014 @08:05AM
(#7005211)
Interesting. Yes, it is very similar in both ruby and python, except that the convention is stronger by virtue of being the default behaviour of #new or .__new__ respectively. The placement of the methods (new on the class object and init[ialize] on the instance) is identical.
I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it directly, though I go out of my way to understand its influences on more contemporary languages.
Re:Making Python's __init__ method magical
by reuven on Friday January 03, 2014 @09:03AM
(#7005426)
My very limited experience working with the Smalltalk language and environment was quite negative, although I hear people sing its praises, so perhaps I just didn't know what I was doing.
That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, and thus get a deeper understanding of how it works.
Re:Making Python's __init__ method magical
by radiowave on Friday January 03, 2014 @09:28AM
(#7005535)
Yes, and while I'm generally of the view that Smalltalk has lessons to teach, of course they aren't all good ones - in particular I sometimes feel that the way construction and initialization is done in Smalltalk has tended to incentivise having getters and setters for every internal variable, and has perhaps contributed to the rather brittle style of program construction that is decried as part of "everything that's wrong with OOP".
Re:Making Python's __init__ method magical
by patrickmay on Friday January 03, 2014 @01:01PM
(#7006877)
Good point. Python's __init__ is closer to Common Lisp's 'initialize-instance :after' except for the fact that Common Lisp provides :initarg to allow make-instance to fully construct an object in most cases.
Re:Making Python's __init__ method magical
by _0ffh on Saturday January 04, 2014 @10:00PM
(#7014288)
So, we've got three roles and three names... just let's call the allocator "allocator", the initializer "initializer", and the thing which does both the "constructor", maybe? :)
Re:Making Python's __init__ method magical
by lukashed on Friday January 03, 2014 @08:19AM
(#7005251)
Here's another magical version that uses a class decorator:
class Autoinit(object):
def __call__(self, cls):
orig_init = cls.__init__
arg_count = orig_init.func_code.co_argcount
arg_vars = orig_init.func_code.co_varnames[1:1 + arg_count]
class Wrapped(cls):
def __init__(self, *args, **kwargs):
for pos, var in enumerate(arg_vars[:len(args)]):
setattr(self, var, args[pos])
for var in kwargs:
setattr(self, var, kwargs[var])
return orig_init(self, *args, **kwargs)
return Wrapped
if __name__ == '__main__':
@Autoinit()
class Test(object):
def __init__(self, a, b):
pass
t = Test(1, b=2)
print t.a, t.b
[Disclaimer: This is my first class decorator. Please correct me if I'm wrong.]
Re:Making Python's __init__ method magical
by mattdeboard on Friday January 03, 2014 @08:37AM
(#7005304)
Oh boy, so I started using Python first. I took "reflection everywhere" for granted so hard without even knowing it. I learned Clojure next, and while it had its rough points I never thought, "Man this is really hard!"
Then, I learned some Scala & Java. That's when I realized how for granted I took this thing called "reflection". Especially when it came to ORMs. Holy mackerel. I got over it but it is definitely a huge thing -- for better and worse -- that is so easy in Python that really spoils beginners.
Re:Making Python's __init__ method magical
by solox3 on Friday January 03, 2014 @10:48AM
(#7005968)
Interesting you mentioned CoffeeScript and syntactic sugar, because adding an equivalent "self." in python inits' parameter names doesn't work the same way in CoffeeScript.
def __init__(self, self.x, self.y): # fail
pass
Did you mention this because there is an existing PEP that proposes this functionality?
Re:Making Python's __init__ method magical
by nadaviv on Saturday January 04, 2014 @04:53PM
(#7013119)
No, I just thought the way CoffeeScript handles it is pretty slick. I looked around, and it seems like proposals to add something like this to Python did come up [1] [2], but weren't very welcomed.
Re:Making Python's __init__ method magical
by krupan on Friday January 03, 2014 @12:42PM
(#7006762)
Seriously, I couldn't finish the article once I saw he wasn't using locals. Is there some hidden gotcha that we are missing here that he's taking care of with his more complicated approach?
Re:Making Python's __init__ method magical
by oinksoft on Friday January 03, 2014 @12:48PM
(#7006798)
I don't like code to casually access the inner dict. Now you have to maintain your `k != 'self'` (ugly in its own right) for each argument you want to exclude from this rule. The explicit version is superior, this is needlessly slow and complicated.
Re:Making Python's __init__ method magical
by thom_nic on Friday January 03, 2014 @10:59AM
(#7006042)
I had a different take on this - it's somewhat inspired by collections.namedtuple but mutable and has some other convenient features for data classes: https://gist.github.com/tomstrummer/8240282
(see the examples at the bottom of the file)
This could also be tweaked to use __dict__ instead of __slots__ so it can act as an expando class which I believe is more similar to what the OP achieves.
Re:Making Python's __init__ method magical
by lmm on Friday January 03, 2014 @09:29AM
(#7005539)
This is pretty cool (reminds me of the scala approach to constructors). But please listen to the author's caveats, and never do something like this in a production system; there are few things worse than magic when you're debugging.
Re:Making Python's __init__ method magical
by reuven on Saturday January 04, 2014 @02:45PM
(#7012561)
The thing with Python is that you don't need to know this stuff to use the language effectively. Yes, after a while, you'll want to dive into the innards -- but they're optional for most people during their initial time learning Python.
It's an interesting balance between one being more magical in its use of the system (inspecting live stack frames) and the other being more magical in appearance (having a function with just pass in it actually do something).
> While many people think of __init__ as a constructor, that’s not really the case. Rather, __init__ is invoked on our new object after it has been created
I'm not aware of a language (though I don't claim to know them all!) in which this is not exactly the definition of a constructor. In C++, `operator new` does the allocation, the constructor is called after. In java it's similar. In ruby it's Class#alloc (which is called by Class#new as roughly `ClassName.alloc.tap{|o| o.initialize(*args); o}`)
So the reader is correct to interpret it as a constructor. Constructors initialize the state of an uninitialized object.
Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem from that. I do think the distinction is meaningful and important in languages that separate these two things out, though.
That said, I'm totally willing to believe that neither they nor I understood the difference between an "allocator" and a "constructor." I'll check into this further, both for myself and for the sake of future writing and lectures!
In C++ the heap object initialization goes like this:
This calls either `Obj::operator new` or the globally scoped `::operator new`, which somehow obtains an area of memory at least sizeof(obj) bytes long and returns it. After that calls to the object's constructors are generated and called.`operator new` is an allocator and the constructor initializes the raw bytes returned from it.
You can actually do the process explicitly in its entirety like so using placement new, which is a special form that calls the constructors on an arbitrary location:
You can't call constructors directly (except with a C++0x feature that lets you call them from a same-level constructor) because of the issue of how they're ordered based on the object graph. They have a precondition of their parent class having been fully constructed.[1] Note: I don't really mean this as a judgement, understanding of C++ internals are pretty poor in a lot of programmers and for pretty good reasons a lot of the time.
I am not trying to argue (on the contrary, I would defer to your expertise), but rather I mean to add a small data point of what perspective developers with my background may have.
I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it directly, though I go out of my way to understand its influences on more contemporary languages.
That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, and thus get a deeper understanding of how it works.
Then, I learned some Scala & Java. That's when I realized how for granted I took this thing called "reflection". Especially when it came to ORMs. Holy mackerel. I got over it but it is definitely a huge thing -- for better and worse -- that is so easy in Python that really spoils beginners.
You do see it used for mocking, ORMs, dependency injection, etc.
[1] https://mail.python.org/pipermail/python-dev/2005-July/05456...
[2] http://bytes.com/topic/python/answers/101594-__autoinit__-pr...
(see the examples at the bottom of the file)
This could also be tweaked to use __dict__ instead of __slots__ so it can act as an expando class which I believe is more similar to what the OP achieves.
I'm not sure you and I have the same meaning for that word.