Pro tools like Omnigraffle handle that sort of thing. Best of both worlds.
And omnigraffle can take a dot definition as input and the let you fiddle with it manually, it is really a joy to work with.
yEd is freeware (not free software) but I have not found a better program that matches accessibility (Java-based so runs cross-platform) and sophistication in automated graph layouts.
As a result, for either high-speed or larger scale usage, my experience is that text (in combination with constraint-respecting automatic layout options that suit one’s personal aesthetic) really starts to shine.