Rust Dataframe: Update 1

Re:Rust Dataframe: Update 1 by bobbylarrybobby on Sunday April 05, 2020 @12:13PM (#22786413)

You are right about the `out` argument, I'd forgotten about that. But even avoiding the wasteful memory allocations, numpy still is about 60% slower than Julia, as it makes multiple passes over the input data. (If there's a way to get numpy to just make a single pass over the data and remain performant, I'd love to know.)

[ Parent | on HN ]

Re:Rust Dataframe: Update 1 by a_t48 on Sunday April 05, 2020 @01:33PM (#22786958)

The first thing I'd reach for is to refactor into a list comprehension. Looks like this is the proper way to do it:

  for x in np.nditer(a, op_flags = ['readwrite']):
    x[...] = cos(sqrt(sin(pow(x, 2))))

That's some knarly syntax though. I've never seen that ellipsis operator?

Edit: just read about `Ellipsis`. I'm a fan, even if it's sort of nonstandard across libraries. Those readwrite flags are a travesty though, but at least you can paper over it with a helper function.

Something like:

  def np_apply(a, f):
    for x in np.nditer(a, op_flags = ['readwrite']):
      x[...] = f(x)

  np_apply(x, lambda x: return cos(sqrt(sin(pow(x, 2))))))

  def np_apply(a, *argv):
    for x in np.nditer(a, op_flags = ['readwrite']):
      for f in argv:
        x = f(x)
      x[...] = x

  np_apply(lambda x: return pow(x, 2), sin, sqrt, cos)

Edit3: There's a way to turn this into "pythonic" list comprehension code, but it would probably only make it look prettier rather than more performant.

[ Parent | on HN ]