News for hackers. Stuff that matters.
Re:D3 3.3 Released by javajosh on Thursday August 22, 2013 @03:00PM (#6259270)
I find D3 to be frustrating because it promises so much and yet it's less of a library and more of a "way to write JavaScript". There really isn't an abstraction so much as a convention for calculating the 3 subsets of an intersection and associating each subset with DOM elements, and then 'doing something' with those DOM elements, possibly as a function of the associated data item. Mike has created some amazing visualizations with D3, but for each viz the code is fairly long and hairy. The worst thing though is that it's not composable - or rather, the unit of composability is very low level. D3 doesn't have any helpers for data that changes over time, for example.
Re:D3 3.3 Released by ganeumann on Thursday August 22, 2013 @03:39PM (#6259542)
As a programmer I'm a hacker--in the golf sense of "someone who spends a lot of time at something but still sucks." But d3 has allowed me to build several useful data visualizations. The code may not be short and sweet, but it is pretty easy to write and pretty easy to understand. Perfect for someone looking for just the right amount of abstraction.

Here's something I wrote a while ago using d3 for my own use. It's entirely client-side js. http://neuvc.com/labs/vcbar/ It charts the investment activity of VCs by pulling Crunhbase data. I would not have written this if I didn't have d3 to use as a crutch for the bar charts.

Re:D3 3.3 Released by JungleGymSam on Thursday August 22, 2013 @05:41PM (#6260285)
> but it is pretty easy to write and pretty easy to understand

I wish I was as smart as you because D3 makes almost zero sense to me. I think there's something I'm missing that, once it clicks, will change things. <fingers crossed/>

Re:D3 3.3 Released by ganeumann on Thursday August 22, 2013 @07:16PM (#6260723)
It's not a question of smartness. Take a working example of code and make it do something else. I'm sure I took some bar chart code from one of Bostock's examples and used that as a base. After a few hours of struggling with it (and with freakin' Javascript) and trial and error debugging, it became clear to me what d3 wanted me to do. Then it was easy.
Re:D3 3.3 Released by jamesjporter on Thursday August 22, 2013 @10:42PM (#6261396)
In my experience D3 is very much like that. Before you wrap your head around the way its data binding model works it seems totally arcane (esp. if you are like me coming from using other visualization tools, like matplotlib, where the hello world example is just "plot(x,y)"). Keep banging your head against the wall, it will eventually crumble and all will become clear.
Re:D3 3.3 Released by coldtea on Thursday August 22, 2013 @05:47PM (#6260313)
The fact that it's like a functional set of transformations, using closures?

Perhaps you're thinking to imperatively. Read a simple D3 tutorial and things will click.

Re:D3 3.3 Released by kmfrk on Thursday August 22, 2013 @03:09PM (#6259354)
I run into A LOT of problems trying to make my own charts, but it also serves to show in many cases that you can't abstract charting to account for all the use cases, because there are so many distinct characteristics in every graphic and dataset.

If you want a higher level of abstraction for d3, try Chartbuilder, which I find doesn't cover anywhere near my use cases: https://github.com/Quartz/Chartbuilder.

Another tool is Bret Victor's proof-of-content editor: http://worrydream.com/#!/DrawingDynamicVisualizationsTalk.

Re:D3 3.3 Released by kmfrk on Thursday August 22, 2013 @06:40PM (#6260557)
By the by: Magic Maps is unbelievably awesome to get your map nerd on. (Or playing around with any shapefile, really.)

Check it out at http://magicmaps.evanmiller.org/gallery.html.

Re:D3 3.3 Released by csmatt on Thursday August 22, 2013 @04:37PM (#6259898)
Did he release a tool or source for what he uses in the video?
Re:D3 3.3 Released by kmfrk on Thursday August 22, 2013 @04:45PM (#6259949)

    I expect to write a full research report at some point 
    (at which I'll make the research prototype available as
    well).
http://worrydream.com/DrawingDynamicVisualizationsTalkAddend...
Re:D3 3.3 Released by javajosh on Saturday August 24, 2013 @02:01PM (#6269297)
Bret Victor is famous for tantalizing people with the future but not releasing code :)
Re:D3 3.3 Released by kmfrk on Saturday August 24, 2013 @02:08PM (#6269322)
Yeah, I'm not holding my breath either. I think his prototypes are usually just conceptual and not meant for use cases, which makes his talks a bit too unfair and utopian for my taste.
Re:D3 3.3 Released by pessimizer on Thursday August 22, 2013 @04:06PM (#6259701)
I think the secret to d3 is that before you can get much out of it, you basically have to be comfortable with hand-writing static visualizations in SVG. Once you can do that, d3 can give them life.
Re:D3 3.3 Released by uptown on Thursday August 22, 2013 @05:21PM (#6260173)
I've found D3 to be tremendously powerful for some types of visualization. For more-traditional types of plots, I usually use Flot instead. http://www.flotcharts.org/
Re:D3 3.3 Released by fletchowns on Friday August 23, 2013 @01:02AM (#6261805)
http://www.highcharts.com/ is worth mentioning also, really awesome high quality charting library.
Re:D3 3.3 Released by coldtea on Thursday August 22, 2013 @05:45PM (#6260301)
>Mike has created some amazing visualizations with D3, but for each viz the code is fairly long and hairy.

Huh? I find D3 code to create a visualisation be much much smaller when comparing with competing implementation of the same visualization.

But then, my metric of comparison is creating visualisations from scratch with imperative drawing APIs (on Canvas/SVG).

Perhaps you want a one-line function invocation to draw the whole thing?

D3 is not for that, it's a plotting HELPER library, not an ready-made collection of plots.

That said, it's not difficult to adapt individual plots made with it.

Re:D3 3.3 Released by javajosh on Thursday August 22, 2013 @10:08PM (#6261310)
I'm comparing D3 with two of it's progenitors, Prefuse and Flare. All of this is coming out of Prof. Jeff Heer's viz group at Berkeley.
Re:D3 3.3 Released by th0ma5 on Thursday August 22, 2013 @03:37PM (#6259529)
Thank you for posting this. I had a lot of issues with D3 where it seemed like it wanted to be a domain specific language, but at some point decided to be a reimplementation of JS around some arbitrary starting point of thinking in R-like or HDF-like data frames or something, and then just got out of hand.

There are a ton of cool interpolation and features and such, and you can use them probably here and there without everything else from D3, and I have managed to do that on occasion. If someone were to make a debugger specifically for D3 or just generally say "forget JavaScript and drink the coolaid and just copy our awesomeness and forget about making your own" I think that would be in line with how D3 is presented. The composable part is spot on. I think OpenFrameworks also suffers from this. They demo their awesome thing and eventually make a library in OF, but as far as using OF itself to make that original awesomeness, that's when one would just say "well, I guess OF is cool for doing exactly what you see people doing with OF but anything more is just C++" ... anyway. So it isn't a framework or an abstraction per se, but a series of just previously developed functions that let you repeat what the author did.

Re:D3 3.3 Released by smrtinsert on Thursday August 22, 2013 @04:55PM (#6260009)
Personally I feel the reach is too broad to be considered a library. It's becoming some of an Ext for visualizations.
Re:D3 3.3 Released by oscilloscope on Thursday August 22, 2013 @04:13PM (#6259759)
D3 is definitely a library, though not a framework. There are conventions, but there are also magical abstractions that work together like selections, data-binding and transitions. I would not call the eight lines of code you need to write for a basic visualization and transition "low-level".

Composability is achieved with configurable functions. Is there an alternate solution you propose? What helper would be useful for data that changes over time?

Re:D3 3.3 Released by smrtinsert on Thursday August 22, 2013 @04:57PM (#6260020)
svg.js is a library. d3 is definitely a framework.
Re:D3 3.3 Released by roycoding on Thursday August 22, 2013 @06:22PM (#6260483)
This is an interesting comment to me because I came to D3 having written zero Javascript and picked it up as I learned D3. D3 does feel like both a library plus a set of conventions. That being said, I find it to be very powerful once you grasp the basics. I've also used some higher level libraries like NVD3.js when I didn't feel like starting from scratch.
Re:D3 3.3 Released by mparr4 on Thursday August 22, 2013 @01:04PM (#6258367)
D3 is an absolute blast. I'm continually impressed by it's accessibility and flexibility. If you've never played with it before it's an absolute joy to work with.

Make sure to check out Mike Bostock's (d3's primary author) absurdly large collection of examples: http://bl.ocks.org/mbostock

Re:D3 3.3 Released by car on Thursday August 22, 2013 @07:41PM (#6260791)
I second this sentiment. Transform based zoom is a great new feature, now I don't have to roll my own.
Re:D3 3.3 Released by dljsjr on Thursday August 22, 2013 @02:52PM (#6259202)
I've never had the opportunity to do much with D3 myself, but one of the things that strikes me as really neat about D3 is that the data visualization is aesthetically pleasing (use of colors, transparency, stroke weight, etc.). I love to just sit and look at D3 documents because I often find them to be very soothing and enjoyable as visual elements, independent of the data being presented.

Conversely, I very rarely see any demonstrations of non-Web plotting/visualization tools (MATLAB, matplotlib, gnuplot, etc.) that look nearly as nice.

What's the reason for this? Does D3 look that nice out of the box, or are D3 users/demonstrators just more interested in flipping the appropriate bits and writing the code and doing the work to make sure that their data looks nice? I ask because I'm about to start playing with a personal project that isn't web based and that needs plotting. I'm seriously contemplating embedding a WebKit panel and using D3 to render the graphics, but I haven't had the chance to really invest a lot of time in studying plotting options. D3 has definitely given the best first impression though.

Re:D3 3.3 Released by kmfrk on Thursday August 22, 2013 @03:03PM (#6259303)
Mike Bostock has a great design sensibility, but there is also some work behind the scenes. It's a little like the LaTeX of charts that way.

Notice how the ticks look, which is dynamically handled, for instance. There's some tweakage behind how many ticks to display, where to place ticks, and other Tuftian goodness.

Re:D3 3.3 Released by mschulkind on Thursday August 22, 2013 @02:57PM (#6259251)
D3 is not a graphics library, so it's certainly not D3 itself making anything look nice. D3 enables you to easily build great visualization, but it doesn't actually draw anything directly or provide you with a theme or anything like that.

D3 pure and simple binds data to DOM elements and keeps the relationship updated, but nothing more. Those DOM elements could be pure text, or tables, or whatever else you want. It just so happens that D3 works well when bound to SVG elements, but that's purely up to you.

Re:D3 3.3 Released by bdon on Thursday August 22, 2013 @03:33PM (#6259509)
A few reasons:

D3 has built-in components to draw small details like axes precisely, which lower the barrier to making good data graphics.

Also, the feedback loop in my experience is much tighter than with tools like Processing, R or Matlab, since you can manipulate elements directly in your browser, instead of having to compile code/generate a new raster image every time.

Re:D3 3.3 Released by subbu on Friday August 23, 2013 @06:59AM (#6262714)
As someone already said, Mike Bostock has a great design sense. But more importantly, he has built up a massive set of ready-to-use visualizations. Most others, like me, use his examples as a base to build our visualizations. We often reuse the same properties used in the examples. For example, most circles have a contour of steelblue with a stoke width of 1.5 pixels, text nodes are all sans-serif of 10 pixels, all paths (lines between nodes) are colored #ccc with a stroke width of 1.5 pixels. These are pretty good defaults.
Re:D3 3.3 Released by jws on Thursday August 22, 2013 @01:05PM (#6258378)
What is D3?

D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.

Re:D3 3.3 Released by ibudiallo on Thursday August 22, 2013 @01:24PM (#6258517)
I was completely lost without your comment. Thank you
Re:D3 3.3 Released by groovy2shoes on Thursday August 22, 2013 @02:07PM (#6258847)
I'm still completely lost. How is a "document based on data" different from any other document?
Re:D3 3.3 Released by mbostock on Thursday August 22, 2013 @02:35PM (#6259069)
The “document” here refers to the W3C Document Object Model (DOM): the hierarchy of HTML or SVG elements that comprises a webpage. And “data” refers to arbitrary JavaScript data structures, like an array of numbers or a parsed CSV file. So, D3 is a JavaScript library for transforming the DOM based on data.
Re:D3 3.3 Released by mparr4 on Thursday August 22, 2013 @02:20PM (#6258949)
http://mbostock.github.io/d3/tutorial/circle.html

The above is an excellent example of both what d3 is AND how it works.

Re:D3 3.3 Released by captaincrowbar on Thursday August 22, 2013 @06:53PM (#6260616)
Seems to be broken on Chrome (Win7 x64). Clicking on any of the Run buttons does nothing.
Re:D3 3.3 Released by BHSPitMonkey on Thursday August 22, 2013 @08:30PM (#6260984)
"Data" as in "datasets", not "paragraph data" or "JPEG data".
Re:D3 3.3 Released by crm416 on Thursday August 22, 2013 @01:20PM (#6258487)
Highly recommend D3. I've used it for a large-scale project this summer, handling thousands (millions, even) of data points in the browser. Mike Bostock has created an immense body of examples and is constantly found answering questions on SO, etc. Kudos to Mike and D3 as a whole.

EDIT: oops, typo

Re:D3 3.3 Released by bunkat on Thursday August 22, 2013 @01:24PM (#6258516)
His name is actually Mike Bostock, not Mark :)
Re:D3 3.3 Released by danso on Thursday August 22, 2013 @01:10PM (#6258417)
Not a D3 expert myself so hard to know how "big" each of the bullet points are...but the interpolated zoom is pretty slick: http://bl.ocks.org/mbostock/6301817

And custom geometry transforms: https://github.com/mbostock/d3/wiki/Geo-Streams#wiki-transfo...

Re:D3 3.3 Released by cscheid on Thursday August 22, 2013 @02:09PM (#6258864)
The slickest part of that demo is the continuous level-of-detail rendering. That's extremely helpful when scaling up to large maps.
Re:D3 3.3 Released by oscilloscope on Thursday August 22, 2013 @04:16PM (#6259773)
Brush transitions are a nice feature http://bl.ocks.org/mbostock/6216724
Re:D3 3.3 Released by aviswanathan on Thursday August 22, 2013 @01:06PM (#6258392)
D3 is the reason I got into animation in the first place. The general ease-of-use combined with the hackability of it is awesome.
Re:D3 3.3 Released by sophacles on Thursday August 22, 2013 @01:07PM (#6258393)
D3 is great - It has made a lot of things super easy to make pretty (mostly pretty realtime front ends to research demos). I wish they would add some more support for canvas drawing, but other than that, everything about it is just slick.
Re:D3 3.3 Released by npalli on Thursday August 22, 2013 @01:28PM (#6258546)
What is your specific use case for using Canvas along with D3?
Re:D3 3.3 Released by sophacles on Thursday August 22, 2013 @02:03PM (#6258820)
Right now everything I want to do is capable of being done reasonably well in SVG, however, there has been talk of effects that we wish to add that are easier done in canvas.

We are doing a graph diagram using force directed layout, and are displaying various overlays of paths through the graph based on input data parameters. As I mentioned, this is currently all fine in SVG, however, we'd like to in the future be able to make those overlays a bit more dynamic in the graphic sense - by adding some effects that canvas makes easy. Also, our future drawing of the graph will include enough custom shapes and images/sprites that the canvas context + drawing objects will result in cleaner code than the equivalent SVG operations.

Also, and this part is a personal bias, I just dislike SVG and it's DOM for runtime graphics manipulation, and prefer the canvas API for it.

Re:D3 3.3 Released by npalli on Thursday August 22, 2013 @02:49PM (#6259185)
Thanks for the detailed comment. As you might imagine, I think the killer feature of D3 is the binding of data to the DOM and all the awesome manipulation that is possible (via CSS/javascript/jquery etc.). It also happens to scale very well for us. So I look for examples where Canvas is a better fit.
Re:D3 3.3 Released by ph0rque on Thursday August 22, 2013 @02:32PM (#6259051)
We are doing a graph diagram using force directed layout...

Can you provide a link to this?

Re:D3 3.3 Released by sophacles on Thursday August 22, 2013 @08:14PM (#6260936)
Sorry, can't - it's internal only right now. But the demo from the D3 page was our starting point.
Re:D3 3.3 Released by shabble on Thursday August 22, 2013 @10:18PM (#6261330)
http://bl.ocks.org/rkirsling/5001347 is a cute example I came across recently, if I'm understanding the description correctly.
Re:D3 3.3 Released by crm416 on Thursday August 22, 2013 @01:39PM (#6258625)
Seconded--canvas support would be great. I too had to mix D3 with some canvas drawing. The specific use case was that I had a number of diagrams, most of which were handled by D3, but one of which required pixel-by-pixel rendering (i.e., each pixel in the canvas was colored according to some data).
Re:D3 3.3 Released by enoex1 on Thursday August 22, 2013 @02:42PM (#6259115)
canvas can better handle drawing thing which involve tons of objects, as you're working with pixels instead of DOM elements. here's an example of canvas being used for parallel coordinates: http://bl.ocks.org/syntagmatic/2420080
Re:D3 3.3 Released by przemoc on Thursday August 22, 2013 @12:55PM (#6258312)
A bit off-topic, but it bothers me (positively). How in GitHub you can provide that kind of tag summary? Is it part of tag message? (There are even links, so I doubt it.)
Re:D3 3.3 Released by bullfight on Thursday August 22, 2013 @12:58PM (#6258326)
This is a release, a relatively new github feature

https://github.com/blog/1547-release-your-software

Check out the list of D3 releases https://github.com/mbostock/d3/releases/

Re:D3 3.3 Released by przemoc on Thursday August 22, 2013 @01:01PM (#6258347)
Thanks, somehow I overlooked that.

> You can also attach binary assets (such as compiled executables, minified scripts, documentation) to a release.

Nice. I knew they didn't turn Downloads off for good (API still works, or at least worked some time ago), but I didn't expect such move.

Re:D3 3.3 Released by mbostock on Thursday August 22, 2013 @12:58PM (#6258332)
Re:D3 3.3 Released by jsnk on Thursday August 22, 2013 @12:59PM (#6258338)
Are you asking about github releases? https://github.com/blog/1547-release-your-software
Re:D3 3.3 Released by jbmartin on Thursday August 22, 2013 @03:11PM (#6259367)
What's new in 3.3?

- Brush transitions http://bl.ocks.org/mbostock/6216724

- Tools to help dynamic simplification (scale appropriate geometry on zoom): http://bl.ocks.org/mbostock/6252418

- Bug fixes

- Others?