Show HN: Plotting 3 years of hourly data in 150ms

Re:Show HN: Plotting 3 years of hourly data in 150ms by leeoniya on Friday May 01, 2020 @02:46PM (#23045226)

Unfortunately, I have to use Google Analytics and Google Ads at work every day, and these UIs have absolutely terrible performance :(

A small part of the problem is drawing the trend charts. So I decided to make uPlot [1] to see what was really possible.

[1] https://github.com/leeoniya/uPlot

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by contravariant on Friday May 01, 2020 @03:26PM (#23045600)

Honestly the title doesn't do it justice. For a plotting tool it's not an impossible task, but making a webpage this responsive is incredibly hard, so well done there.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by leeoniya on Friday May 01, 2020 @03:29PM (#23045627)

it's an understatement for sure :)

a lot of time is spent on just DOM layout and JITing the JS, which get amortized with huge datasets. the 166k bench doesnt take any longer.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by simmers on Friday May 01, 2020 @05:08PM (#23046511)

Kudos for directing folks to other projects if uPlot isn't fast enough for them. Every developer should take the time to write out appropriate and inappropriate use-cases on the readme.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by doublesCs on Friday May 01, 2020 @03:43PM (#23045768)

Hi, I'm very interested in this. I've been using dask/plotly for a side project but it's just too slow. If you know dask, do you think you could discuss which features of dask your project has and doesn't have?

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by jparmer on Friday May 01, 2020 @04:01PM (#23045956)

Here's a Dash app that use Dask & Datashader for fast aggregations on +40M rows: https://dash-gallery.plotly.host/dash-world-cell-towers

The source code, which can be used as a Dash + Dask boilerplate, is here: https://github.com/plotly/dash-world-cell-towers

Feel free to ask questions at community.plot.ly

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by vamega on Friday May 01, 2020 @04:50PM (#23046335)

I don't think that actually plots the 40M points, just aggregates them in Python and plots a small section of that data on the website?

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by kylebarron on Friday May 01, 2020 @06:35PM (#23047237)

I believe so. An optimization of Datashader is that it _doesn't_ plot all the points, but rather optimizes which points actually influence pixel values.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by gesman on Friday May 01, 2020 @07:40PM (#23047636)

It plots cell towers in the middle of my residential neighborhood where i absolutely sure there aren't any

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by jcims on Saturday May 02, 2020 @01:42AM (#23049574)

https://community.opencellid.org/t/range-of-a-tower/370/2

>The range field tells you the approximate area within which the cell could be, in metres. This is effectively an estimate of how accurate the location is.

There's one marked in the middle of a corn field near me with a range of 48,000m. ?

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by leeoniya on Friday May 01, 2020 @07:43PM (#23047656)

maybe they're Stingrays ;)

https://en.m.wikipedia.org/wiki/Stingray_phone_tracker

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by gesman on Saturday May 02, 2020 @02:23AM (#23049768)

Maybe. However cellmapper.net shows few cell towers where they actually exists and this map doesn't show them.

Credibility?

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by chrissnell on Saturday May 02, 2020 @12:28AM (#23049197)

This is awesome. I wrote some software to pull readings off my weather station and stream them to the browser. The plotting library that I use is pretty awful. Besides being slow, it's constantly hosing up the numbers on the y-axis. I want to give uPlot a shot.

Station: https://carlisleweather.com

Software: https://github.com/chrissnell/gopherwx

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by hardwaresofton on Friday May 01, 2020 @03:10PM (#23045469)

Hopefully the upvotes and being on the front page is enough, but if not just want to say props, this is amazing. Already had it saved in my bookmarks glad to see it pop up again :)

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by leeoniya on Friday May 01, 2020 @03:24PM (#23045586)

thanks!

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by maxwellg on Friday May 01, 2020 @06:32PM (#23047218)

The 166k benchmark is ridiculously snappy. Could you shave off a few more ms by inlining the JSON as a data block?

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by leeoniya on Friday May 01, 2020 @07:17PM (#23047493)

probably, but something about diminishing returns...:p

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by jedberg on Friday May 01, 2020 @06:30PM (#23047200)

This is fantastic and may solve a real problem for me. Thank you so much, and thank you for the MIT license!

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by adeelk93 on Friday May 01, 2020 @05:32PM (#23046737)

I like this a lot! Reminds me of dygraphs, and then I saw it in that in the acknowledgements :)

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by victorthehuman on Sunday May 03, 2020 @06:53AM (#23058222)

This is a test reply please ignore it, it will be deleted in a few minutes.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by hardlianotion on Friday May 01, 2020 @04:36PM (#23046215)

That's rather beautiful.

[ Parent | on HN ]

Re:Show HN: Plotting 3 years of hourly data in 150ms by jrumbut on Saturday May 02, 2020 @02:32AM (#23049808)

This is just incredible.

[ Parent | on HN ]