News for hackers. Stuff that matters.
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by zylepe on Saturday May 02, 2020 @12:45PM (#23052962)
Very cool! I’ve done visualizations on similar size datasets by mapping tsne output to fake latitude/longitude coordinates and rendering it with mapbox gl. It feels like a hack, but then you get all of the panning/zooming interaction, styling, and rendering labels with collision avoidance for free. For smaller datasets I pass in geojson directly, but for larger ones I precompute vector tiles so you can lazy-load more of the data when you zoom in. There are a lot of things built for mapping that are relevant for other domains, I wish that there was a common base that could be extracted that mapping libraries adapt to geographic data but data visualizations could adapt to non-geographic data.

I’ll have to play around with your technique more, encouraging to see you get good performance eagerly loading a million data points!

Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by 7373737373 on Saturday May 02, 2020 @02:37PM (#23053885)
I can recommend LargeVis[0][1] which I used to layout and Polymaps[2] to visualize tiles of the https://wikiscape.org article point cloud.

[0] https://github.com/lferry007/LargeVis [1] https://github.com/elbamos/largeVis [2] http://polymaps.org/

Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by 7373737373 on Monday May 11, 2020 @05:22AM (#23140309)
Scrap polymaps, use leaflet instead: https://leafletjs.com
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by andybak on Saturday May 02, 2020 @01:19PM (#23053273)
Deck.GL has some similar examples. I think the reason this makes sense is that geo data viz had to pioneer huge datasets - which are less common in other fields.
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by 1wheel on Saturday May 02, 2020 @03:51PM (#23054395)
Nice write up Colin!

fwiw quadtrees don't make much sense when you're showing a tooltip for ~1 million points: https://roadtolarissa.com/scan-sorted/

Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by ColinEberhardt on Sunday May 03, 2020 @03:53AM (#23057567)
That's really interesting, thanks for sharing those results. So easy to go down the route of premature optimisation!
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by paulgb on Saturday May 02, 2020 @01:00PM (#23053103)
Scott Logic is probably the highest-quality company technology blog I've encountered in a while. I keep returning to it for WebAssembly and WebGL topics and it is consistently high quality. I'm not even sure what Scott Logic does, but they sure know their stuff :)

Kudos, Colin and team.

Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by ColinEberhardt on Sunday May 03, 2020 @04:00AM (#23057599)
Thanks Paul, really appreciate that :-)

Just FYI, we're a medium sized software consultancy based in the UK. We do quite a lot of work in financial services, which is why you'll find quite a few posts on performance and cutting-edge web technology. I started the company blog 12 years ago and now we have ~50 authors creating some amazing content. Unfortunately I don't have the time to write quite as much as I used to!

Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by onion2k on Saturday May 02, 2020 @03:55PM (#23054420)
Their head office is round the corner from my office in Newcastle, UK, and I occasionally bump in to their devs in the local pub and at the JS meetup. They're very good at what they do and lovely people too.
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by luhem7 on Saturday May 02, 2020 @03:56PM (#23054432)
Re:Show HN: Rendering One Million Datapoints with D3 and WebGL by gdubs on Saturday May 02, 2020 @03:06PM (#23054087)
The scatterplot of the library data is fascinating — looks like an organism.