You should check out https://github.com/IGITUGraz/live-plotter (I am the author of the package)
It is designed exactly for this use-case -- visualizing results of an ongoing simulation. It does best effort plotting, but doesn't block computation for plotting. I would be happy to receive feedback on the package.
Cool, thanks.
Re:Python Graph Gallery
by MattConfluence on Monday October 16, 2017 @09:36AM
(#15483041)
What I did when I had a similar problem was to have a completely separate visualization program and pass data messages to it (using Celery and RabbitMQ). But that was in a setting where I was already using Celery for other tasks. Setting up a message queue for just visualization feels a bit over-engineered. Maybe you can just write to a text file, or a Sqlite db, and have the visualization program poll for changes.
Re:Python Graph Gallery
by oneweekwonder on Monday October 16, 2017 @10:06AM
(#15483268)
Rabbitmq or redis will already be in apt or yum so a simple
sudo apt-get install rabbitmq-server # or redis
pip install celery
from celery import Celery
app = Celery('tasks', broker='pyamqp://guest@localhost//')
@app.task
def add(x, y):
return x + y
[0]: http://docs.celeryproject.org/en/latest/getting-started/firs...
Bokeh supports streaming data to existing graphs. This lets you define a graph, display it in a browser, and then push updates from your code. This is a lot lighter than doing a full draw in matplotlib, plus works over the web.
Here's an intro blog post from 2013 [0] and a current tutorial [1]. This functionality depends on Bokeh Server, docs here [2]
[0] https://www.anaconda.com/developer-blog/painless-streaming-p...
[1] http://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blo...
[2] https://bokeh.pydata.org/en/latest/docs/user_guide/server.ht...
coincidentally: https://news.ycombinator.com/item?id=15483264
Oh that sounds excellent, I'll have to check it out.
Veusz (https://veusz.github.io/) does this. Data are sent to the plotting process using pickle over a socket.
I find that for some applications matplotlib really slows down my loops, but there are times that I really need to visualize the in-progress computations. While some other libraries might be faster, what I really want is simply to send instructions and data to a separate process that is performing the plotting. I'd like it to completely render a frame and simply skip instructions for following frames until it's done rendering, avoid buffering up every single frame that is sent to it, and have it plot the next available frame when it's done. I have yet to see a library that functions this way. Basically I want to be able to easily visualize an on-going computation without blocking it just for the rendering of temporary results.