Re:Show HN: Rendering One Million Datapoints with D3 and WebGL
by zylepe on Saturday May 02, 2020 @12:45PM
(#23052962)
Very cool! I’ve done visualizations on similar size datasets by mapping tsne output to fake latitude/longitude coordinates and rendering it with mapbox gl. It feels like a hack, but then you get all of the panning/zooming interaction, styling, and rendering labels with collision avoidance for free. For smaller datasets I pass in geojson directly, but for larger ones I precompute vector tiles so you can lazy-load more of the data when you zoom in. There are a lot of things built for mapping that are relevant for other domains, I wish that there was a common base that could be extracted that mapping libraries adapt to geographic data but data visualizations could adapt to non-geographic data.
I’ll have to play around with your technique more, encouraging to see you get good performance eagerly loading a million data points!
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL
by andybak on Saturday May 02, 2020 @01:19PM
(#23053273)
Deck.GL has some similar examples. I think the reason this makes sense is that geo data viz had to pioneer huge datasets - which are less common in other fields.
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL
by paulgb on Saturday May 02, 2020 @01:00PM
(#23053103)
Scott Logic is probably the highest-quality company technology blog I've encountered in a while. I keep returning to it for WebAssembly and WebGL topics and it is consistently high quality. I'm not even sure what Scott Logic does, but they sure know their stuff :)
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL
by ColinEberhardt on Sunday May 03, 2020 @04:00AM
(#23057599)
Thanks Paul, really appreciate that :-)
Just FYI, we're a medium sized software consultancy based in the UK. We do quite a lot of work in financial services, which is why you'll find quite a few posts on performance and cutting-edge web technology. I started the company blog 12 years ago and now we have ~50 authors creating some amazing content. Unfortunately I don't have the time to write quite as much as I used to!
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL
by onion2k on Saturday May 02, 2020 @03:55PM
(#23054420)
Their head office is round the corner from my office in Newcastle, UK, and I occasionally bump in to their devs in the local pub and at the JS meetup. They're very good at what they do and lovely people too.
I’ll have to play around with your technique more, encouraging to see you get good performance eagerly loading a million data points!
[0] https://github.com/lferry007/LargeVis [1] https://github.com/elbamos/largeVis [2] http://polymaps.org/
fwiw quadtrees don't make much sense when you're showing a tooltip for ~1 million points: https://roadtolarissa.com/scan-sorted/
Kudos, Colin and team.
Just FYI, we're a medium sized software consultancy based in the UK. We do quite a lot of work in financial services, which is why you'll find quite a few posts on performance and cutting-edge web technology. I started the company blog 12 years ago and now we have ~50 authors creating some amazing content. Unfortunately I don't have the time to write quite as much as I used to!
;)