News for hackers. Stuff that matters.
Re:Never run ‘python’ in your downloads folder by bobbyi_settv on Sunday August 23, 2020 @06:10PM (#24255721)
Does anyone know of a write-up that describes Tensorflow's import magic? I'd be curious to read about what they're doing and why.
Re:Never run ‘python’ in your downloads folder by codethief on Sunday August 23, 2020 @06:58PM (#24255999)
I wish I had a write-up but here's a starting point at least:

https://github.com/tensorflow/tensorflow/blob/master/tensorf...

Inside the source code files, they then use a decorator (@tf_export) to define where in the package/module hierarchy a function/class should get placed, see e.g.

https://github.com/tensorflow/tensorflow/blob/master/tensorf...

For instance, the write() function in that module actually gets exposed as `tf.summary.write()` even though it's contained in the file tensorflow/python/ops/summary_ops_v2.py.

Needless to say, this even brings auto-completion of an IDE like IntelliJ to its knees at times.

To make things even worse, things like Tensorboard also make use of global state everywhere, see e.g. the function _should_record_summaries_internal() in the above file. But now the question is: Which module's global state does the function actually access if it gets "moved" to a different module by TF? At the end of the day, I gave up hunting down that bug (_should_record_summaries_internal() always returned False, no matter what) and resorted to mocking the function out whenever I used Tensorboard. To do that (i.e. to find its parent module in the first place), I in turn used the `inspect` package.

This is what I meant by "import magic". :)

Re:Never run ‘python’ in your downloads folder by shp0ngle on Monday August 24, 2020 @12:12AM (#24257467)
this hurts.