Re:Hunting Performance in Python Code: Measuring Memory Cons...
by dom0 on Thursday June 01, 2017 @12:24PM
(#14461676)
Couple notes about cProfile
- The slowdown is difficult to predict and somewhat unstable; never compare timings from runs with/without cProfile
- It's a deterministic, wall-clock profiler
- pyprof2calltree & kcachegrind.
- Loading a profile file = arbitrary code execution. Don't ever load a profile file you did not generate yourself.
Re:Hunting Performance in Python Code: Measuring Memory Cons...
by szatkus on Friday June 02, 2017 @03:34AM
(#14468164)
I find SnakeViz much more simple and user-friendly.
Re:Hunting Performance in Python Code: Measuring Memory Cons...
by mherrmann on Thursday June 01, 2017 @11:32AM
(#14461121)
Interesting point, though again much more complex.
[edit] Anyone reading this: Do not discard cProfile like I did. It just let me shave 900 ms off the startup time of my file manager (mentioned in a child comment). Wow!
Re:Hunting Performance in Python Code: Measuring Memory Cons...
by mattmanser on Thursday June 01, 2017 @11:35AM
(#14461152)
I admit they take time up front to learn to use and interpret the results, but it's a valuable skill and worth doing every year or so even on smaller projects to see if there are any easy wins.
Re:Hunting Performance in Python Code: Measuring Memory Cons...
by mherrmann on Thursday June 01, 2017 @11:50AM
(#14461321)
Omg, cProfile just let me shave 900ms off the startup-time of my PyQt-based file manager [1]. I would not have found this that quickly without your hint. Thank you!
[1]: https://fman.io
Re:Hunting Performance in Python Code: Measuring Memory Cons...
by mattmanser on Thursday June 01, 2017 @12:19PM
(#14461621)
No problem!
This was on the front page today, but flagged as a dupe, also an interesting read as it explains about flame graphs too:
I don't use Python any more and have never profiled it, but there seems to be good options:
https://docs.python.org/2/library/profile.html
There are reasons to do what you're doing, e.g. Stack Exchange have a mini-profiler to watch production, but to identify problems in a development environment, profilers are usually quicker/easier once you've learnt to use them. It also helps you identify problems you didn't even know about.