It's a _famously_ hard problem, which is why most of the graph drawing tools you see outside Graphviz are simple force-directed or tree drawing routines.