News for hackers. Stuff that matters.
Re:Streamz: Python pipelines to manage continuous streams of... by cashsterling on Friday March 27, 2020 @01:24PM (#22704824)
Is streamz/CuDF/CUDA a good option for soft real-time processing of high bandwidth sensor data?

For instance, I have electronics board grabbing 10,000 analog voltage samples a second on multiple channels. I want to: - ingest chunks of that data at 100ms intervals (so 1000 samples per interval) - update some plots in a PyQt-GUI at the 100ms interval - and once a second compute a FFT of 10-20 seconds of data for one or more channels of data.

I have basic code working using numpy, numpy.roll().

I am debating whether or not I should be using C#/.NET or C++ instead of python for this... there are strong speed advantages.

Numpy is fast enough to handle what I need to do right now... but if I want to up my data rate by 10X (10kHz to 100kHz) I think I am going to run into hard limitations with python and numpy and PyQt GUI updates.

Thoughts and suggestions are much appreciated :-)

There are no comments on this comment yet.