Re:Use JSON files as if they are Python modules
by orf on Monday January 25, 2016 @04:36AM
(#10966335)
Import hooks are awesome, one of the cool things you can do is re-write the code as you load it. I made a toy loader that automagically inlines (some) Python function calls[1]. There are also cool projects like MacroPy[2] that do much more extensive things.
1. https://tomforb.es/automatically-inline-python-function-call...
Re:Use JSON files as if they are Python modules
by dr_zoidberg on Monday January 25, 2016 @10:12AM
(#10967562)
I read your experiment on inlining and it looks quite interesting. Did you do a benchmark on how it affects (improves?) performance? In my line of work I've found ocasionally places where inlining would've helped (functions that do masking with 64 bit masks, for example, are a mess to "inline by hand" and kill a lot of readability and clarity). Even with the current limitations of your "toy" implementation, it seems like it would help to avoid the costly function calls.
Re:Use JSON files as if they are Python modules
by finyeates on Monday January 25, 2016 @06:01AM
(#10966534)
Awesome, I saw your post a while back and it triggered me to write a full inliner for python, also added some IFDEF macros using context managers e.g.
with IFDEF("DEBUG"):
I'll have to dig around and find my code, your inline project was a great help in understanding the import hooks and walking the ast.
Re:Use JSON files as if they are Python modules
by jimbokun on Monday January 25, 2016 @11:44AM
(#10968172)
Sounds a bit like the Ruby-fication of Python.
Re:Use JSON files as if they are Python modules
by jamesdutc on Monday January 25, 2016 @01:43AM
(#10965944)
A simple example of how import hooks use for module relocation/deprecation:
Re:Use JSON files as if they are Python modules
by Grue3 on Monday January 25, 2016 @07:30AM
(#10966791)
This seems like a huge case of "implicit over explicit". Ideally if you import package foo, it shouldn't affect anything but foo.bar
Re:Use JSON files as if they are Python modules
by icebraining on Monday January 25, 2016 @07:50AM
(#10966862)
Python itself never enforced that, though. It's the libraries' responsibility to have a "register_import_hook()" function or similar instead of just doing so unannounced.
Re:Use JSON files as if they are Python modules
by inglor on Monday January 25, 2016 @05:49AM
(#10966506)
For what it's worth - modern JavaScript does it too, it separates the syntax (import/export) and semantics of binding the reference from the actual loading mechanism. Not only can you "hook" into imports, you can replace them altogether if you'd like.
Re:Use JSON files as if they are Python modules
by bdarnell on Monday January 25, 2016 @11:31AM
(#10968060)
The "loader protocol" (PEP 302) is a related trick that can be used locally, so it only affects imports from packages that opt in. I use this in https://github.com/bdarnell/codegenloader to automagically process protobuf/thrift files and import the generated code.
https://docs.python.org/3/reference/import.html#import-hooks
https://www.python.org/dev/peps/pep-0302/
You can see the import hook added here:
https://github.com/kragniz/json-sempai/blob/master/jsonsempa...
Import hooks are a great feature in Python.
Take the `import` mechanism in the language and reduce it to its barest theoretical formulation - what does it really do? (Name binding.) Think of all the other features in your application that can be reduced to this. (e.g., configuration management) Use the constraints of the import mechanism to guide the design of this feature, and use import hooks to implement it. You'll end up with an implementation that is very "close" to the core language. I would assert that this closeness strongly suggests correctness and composability.
Less philosophically, think of the Python virtual machine as a system with lots of "safety valves" and "escape hatches." Import hooks are one such safety valve. Think about all the things you could do easily by hooking into module importing? (Real use-cases: loading code from bitemporal object databases, deprecation of modules, relocation of modules, configuration management, &c.)
Of course, there are languages that are much more flexible than Python in this respect. Python aims for a practical flexibility, and I find that, in practice, Python strikes a nice balance.