News for hackers. Stuff that matters.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by boltzmannbrain on Thursday June 01, 2017 @11:33AM (#14461136)
Some guidelines I like for Python optimization (and on loops specifically):

- Rule 1: profile first! Only optimize when there is a proven speed bottleneck; I like using kernprof to profile. When you do go to optimize, test it in a tight loop with time.clock().

- Go for built-in functions; you can't beat a loop written in C.

- Use intrinsic operations. An implied loop is faster than an explicit for loop, and a while loop with an explicit loop counter is even slower.

- Avoid calling functions written in Python in your inner loop. This includes lambdas. In-lining the inner loop can save a lot of time.

- Local variables are faster than globals; if you use a global constant in a loop, copy it to a local variable before the loop. And in Python, function names (global or built-in) are also global constants!

- Try to use map(), filter() or reduce() to replace an explicit for loop, but only if you can use a built-in function: map with a built-in function beats for loop, but a for loop with in-line code beats map with a lambda function!

- Check your algorithms for quadratic behavior, but notice that a more complex algorithm only pays off for large N.

More at https://www.python.org/doc/essays/list2str/

Re:Hunting Performance in Python Code: Measuring Memory Cons... by joncampbelldev on Thursday June 01, 2017 @04:30PM (#14464300)
This advice seems to really highlight the difference between high level managed memory languages languages:

those that fallback to native functions for performance Vs those that work on a fast VM / compilation to native code.

it seems like the former gets some quick performance wins but results in being unable to use the language as intended in any remotely hot spots.

Re:Hunting Performance in Python Code: Measuring Memory Cons... by fnord123 on Thursday June 01, 2017 @07:00PM (#14465716)
An almost free FFI to C is one of python's best features. Using this great feature is definitely using it as intended.
Re:Hunting Performance in Python Code: Measuring Memory Cons... by joncampbelldev on Friday June 02, 2017 @10:47AM (#14470267)
Only if every python programmer can easily implement and distribute their own c functions for every performance hotspot they find in their program.

This is the trade-off I was talking about, you cannot trust the language itself to be performant, you have to rely on others or jump through extra hoops. It's certainly one way to ensure you won't spend the time to fix performance issues until they're really biting you (premature optimisation quote goes here).