Re:Profiling Python Like a Boss
by bryanh on Monday December 02, 2013 @05:30PM
(#6837062)
Op here, happy to answer any questions. These tools have helped us many times (from speeding up various Django extensions to some really complex code deep inside Zapier).
[ Parent
| on HN ]
Good stuff. If you're going to indulge the optimization fetish, at least profile first! Corollary: learn to use gprof and/or your language's profiler, it pays off bigtime.
Question. It's been a while since I used them, but what happens when you use cProfile or kernprof in multithreaded / forking code? I vaguely recall wiring up a custom trace event logger via sys.settrace() to work around this.
I haven't a clue! That is an excellent question, most of our multiprocessing is done via some task based & message queue based map-reduce, so we've not needed to do process or thread profiling.