News for hackers. Stuff that matters.
Re:Copying data is wasteful, mutating data is dangerous by mcv on Friday January 10, 2020 @02:09AM (#22008955)
If you need to send a temporary array through a number of operations (normalise, then vusalise, in this case), you could also set up the stream of operations and send the elements of the array through it one at a time, or in short batches.
Re:Copying data is wasteful, mutating data is dangerous by davedx on Friday January 10, 2020 @03:09AM (#22009285)
Streaming and lazy evaluation is definitely a solution here. But do python libraries support it well?
Re:Copying data is wasteful, mutating data is dangerous by mcv on Friday January 10, 2020 @07:12AM (#22010415)
You can't have true lazy evaluation because you still need the maximum and minimum to normalise them. But that's just two fields that you need only once.

I have no idea whether python or any other language supports this out of the box, but I would expect there are libraries that do this, and if not, it shouldn't be too hard to write this yourself. The main thing is probably that you need to separate the normalisation parameters from the normalisation itself, and after that, you can simply do something like (in javascript):

  array.forEach(e => visualise(normalise(e, min, max)));
Of course visualisation is probably going to involve a new datastructure that's going to hold this data in some way, so there's still going to be another copy of that array living somewhere. That's unavoidable.