Re:CometML wants to do for machine learning what GitHub did ...
by henripal on Thursday April 05, 2018 @02:36PM
(#16767685)
Cool.
I'm guessing from trying it out that you're using Highcharts.
I've run into really unpleasant memory leaks/slowness when streaming data (especially when the existing chart is already thousands of data points). Are you seeing something similar?
[ Parent
| on HN ]
Re:CometML wants to do for machine learning what GitHub did ...
by nicodjimenez on Thursday April 05, 2018 @02:41PM
(#16767753)
Yes using Highcharts. You've had issues with Highcharts? Yeah it's not designed to stream data extremely rapidly but it's a great "good enough" product, especially for something like Losswise where the differentiation is the overall design and architecture and developer experience, not the prettiest possible graphs.
Re:CometML wants to do for machine learning what GitHub did ...
by henripal on Thursday April 05, 2018 @02:45PM
(#16767793)
Yes... For example if I'm running three experiments at the same time, auto-refreshing the chart every two seconds, it essentially freezes the app to a crawl after a thousand points or so. So we reverted to manual updates.
If you know of any better alternatives for data streaming, I'm curious. I tried benchmarking a couple libs recently: https://github.com/henripal/ChartingLibBenchmark
Re:CometML wants to do for machine learning what GitHub did ...
by TorsteinHonsi on Friday April 06, 2018 @04:22AM
(#16771784)
As a Highcharts developer, I had a look at your benchmarking, and have some thoughts about optimizing for Highcharts. The first step is to turn off animation, which helps a lot. The default Highcharts animation on addPoint is 250ms, so with a refresh rate of 100ms you will get a lot of redrawing going on for nothing. The second thing that possibly optimizes a bit is to use hard-coded axis values so that it doesn't have to recompute axis values for each iteration.
With those modifications the performance is much better: http://jsfiddle.net/highcharts/1o5ghqc8/
Re:CometML wants to do for machine learning what GitHub did ...
by nicodjimenez on Thursday April 05, 2018 @03:20PM
(#16768087)
Yeah I have no idea. If you need really high performance that Highcharts doesn't provide you probably need to write your own specialized charting library.