Python 2.7.18, the last release of Python 2

Re:Python 2.7.18, the last release of Python 2 by xscott on Monday April 20, 2020 @05:36PM (#22928864)

As soon as you do any operations (addition, subtraction, etc...) on those views, you're going to get temporary arrays.

For instance, your Mandelbrot example doesn't even use views and it creates two temporaries the size of the entire array on each iteration:

    for j in range(255):
        t = z**2    # create a new squared array
        u = t + c   # create a new summed array
        z = u       # replace the old array

And all of this is ignoring how inconsistent numpy is about when it creates a view and when it creates a copy.

[ Parent | on HN ]

Re:Python 2.7.18, the last release of Python 2 by shoyer on Monday April 20, 2020 @09:03PM (#22930752)

Unnecessary temporary arrays is definitely a major source of inefficiency when working with NumPy, but recent versions of NumPy go to heroic lengths (via Python reference counting) to avoid doing so in many cases: https://github.com/numpy/numpy/blob/v1.18.3/numpy/core/src/m...

So in this case, NumPy would actually only make one temporary copy, effectively translating the loop into the following:

    for j in range(255):
        u = z**2   # create a new squared array
        u += c     # add in-place 
        z = u      # replace the old array

[ Parent | on HN ]

Re:Python 2.7.18, the last release of Python 2 by joshuamorton on Monday April 20, 2020 @06:02PM (#22929177)

Your general point is correct, although in this specific instance, replacing the loop body with

    z **= 2 
    z +=c

gets rid of the temps. But yes there are cases where that isn't possible.

[ Parent | on HN ]

Re:Python 2.7.18, the last release of Python 2 by shoyer on Monday April 20, 2020 @09:25PM (#22930893)

This gets rid of temporary arrays, but this still isn't optimal if z is large. Memory locality means it's faster to apply a scalar operation like z2+c in a single pass, rather than in two separate passes.

Explicitly unrolling loopy code (e.g., in pypy or Numba) is one easy way to achieve this, but you have to write more code.

Julia has some really nice syntax that lets you write things in this clean vectorized way but still get efficient code: https://julialang.org/blog/2017/01/moredots/

[ Parent | on HN ]

Re:Python 2.7.18, the last release of Python 2 by joshuamorton on Monday April 20, 2020 @09:41PM (#22931002)

Gotcha. I feel like I remember some numpy or scipy way of creating complex ufunc ops and applying them simultaneously, but maybe I'm misremembering or thinking np.vectorize was fast?

[ Parent | on HN ]