News for hackers. Stuff that matters.
Re:SciPy 1.0: fundamental algorithms for scientific computin... by tda on Tuesday February 11, 2020 @04:10AM (#22297114)
I've had this experience when writing a simple algorithm for run length encoding, a problem which I struggled to vectorize. Compare the numpy algo with pure python:

  def rle_encode_pure_python(sequence):
    """Run length encode array."""
    previous = sequence[0]
    count = 1
    out = []
    for element in sequence[1:]:
        if element == previous:
            count += 1
        else:
            out.append((count, previous))
            previous = element
            count = 1
    out.append((count, previous))
    return out


  def rle_encode_numpy(sequence):
    diffs = np.concatenate(( np.array((True, )), np.diff(sequence)!=0))
    indices = np.concatenate((np.where(diffs)[0],np.array((sequence.size, ))))
    counts = np.diff(indices).astype('uint16')
    values = sequence[diffs].astype('uint8')
    return np.rec.fromarrays((counts, values),names=('count','value'))

The numpy version is faster, but a lot less legible (and makes some assumptions about data types and array length). Luckily the pure python version can be jitted with numba and is faster than numpy.
Re:SciPy 1.0: fundamental algorithms for scientific computin... by enriquto on Tuesday February 11, 2020 @05:00AM (#22297314)
This is a great example, I love it, thank you very much!

Note: the numpy version is missing the import.

Note2: I tend to prefer using numba than numpy. Yet, sometimes numpy is inevitable, especially for linear algebra routines. In that case, I am in a world of pain because numba and numpy do not interact well at all!

Re:SciPy 1.0: fundamental algorithms for scientific computin... by deshpand on Tuesday February 11, 2020 @09:59AM (#22298942)
< numba and numpy do not interact well at all

Can you please elaborate on this? Not challenging you but trying to understand the nuances.

Numba tutorial says otherwise: https://numba.pydata.org/numba-doc/dev/user/5minguide.html "Numba likes NumPy functions"

Re:SciPy 1.0: fundamental algorithms for scientific computin... by jononor on Tuesday February 11, 2020 @04:35AM (#22297206)
Nice and concise example. Would love a mini-writeup / benchmarks for this (including the numba version).
Re:SciPy 1.0: fundamental algorithms for scientific computin... by tda on Tuesday February 11, 2020 @07:35AM (#22297947)
Here is the code in a notebook: https://github.com/Python-Meetup-Rotterdam/meetup1/blob/mast...

Gave a little presentation on my findings on using different approaches

Re:SciPy 1.0: fundamental algorithms for scientific computin... by m4r35n357 on Tuesday February 11, 2020 @05:39AM (#22297464)
Returning a list? Nay, Yield!
Re:SciPy 1.0: fundamental algorithms for scientific computin... by tda on Tuesday February 11, 2020 @07:41AM (#22297979)
Completely agreed, would never write it this way anymore but the example is a bit older