News for hackers. Stuff that matters.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by mattmanser on Thursday June 01, 2017 @11:28AM (#14461078)
I hope I'm not teaching you to suck eggs, but you can use a profiler to do that rather than changing your code with what are basically print statements.

I don't use Python any more and have never profiled it, but there seems to be good options:

https://docs.python.org/2/library/profile.html

There are reasons to do what you're doing, e.g. Stack Exchange have a mini-profiler to watch production, but to identify problems in a development environment, profilers are usually quicker/easier once you've learnt to use them. It also helps you identify problems you didn't even know about.

Re:Hunting Performance in Python Code: Measuring Memory Cons... by dom0 on Thursday June 01, 2017 @12:24PM (#14461676)
Couple notes about cProfile

- The slowdown is difficult to predict and somewhat unstable; never compare timings from runs with/without cProfile

- It's a deterministic, wall-clock profiler

- pyprof2calltree & kcachegrind.

- Loading a profile file = arbitrary code execution. Don't ever load a profile file you did not generate yourself.

Re:Hunting Performance in Python Code: Measuring Memory Cons... by szatkus on Friday June 02, 2017 @03:34AM (#14468164)
I find SnakeViz much more simple and user-friendly.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by mherrmann on Thursday June 01, 2017 @11:32AM (#14461121)
Interesting point, though again much more complex.

[edit] Anyone reading this: Do not discard cProfile like I did. It just let me shave 900 ms off the startup time of my file manager (mentioned in a child comment). Wow!

Re:Hunting Performance in Python Code: Measuring Memory Cons... by mattmanser on Thursday June 01, 2017 @11:35AM (#14461152)
I admit they take time up front to learn to use and interpret the results, but it's a valuable skill and worth doing every year or so even on smaller projects to see if there are any easy wins.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by mherrmann on Thursday June 01, 2017 @11:50AM (#14461321)
Omg, cProfile just let me shave 900ms off the startup-time of my PyQt-based file manager [1]. I would not have found this that quickly without your hint. Thank you!

[1]: https://fman.io

Re:Hunting Performance in Python Code: Measuring Memory Cons... by mattmanser on Thursday June 01, 2017 @12:19PM (#14461621)
No problem!

This was on the front page today, but flagged as a dupe, also an interesting read as it explains about flame graphs too:

https://news.ycombinator.com/item?id=14384370

Re:Hunting Performance in Python Code: Measuring Memory Cons... by mherrmann on Thursday June 01, 2017 @11:38AM (#14461201)
Yes, you're right. Mine is just the simplest and probably least powerful solution. cProfile is definitely more powerful.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by rrauenza on Friday June 02, 2017 @01:59AM (#14467798)
I think yours still has a valid use in production logging...