I think the other desiderata mentioned in the original post are more important than choice of serialization format. SVG already exists, and we're kind of stuck with it. There are other applications where having a better-designed vector graphics format might pay off, and tying it to the browser all over again wouldn't be a win.

At that point, using an s-expression format (x-expression, EDN, I'm agnostic) should be the first choice. One of the first things that would have to be implemented, of course, is a translator to SVG.

My interest in vector graphics is such that, on the few occasions I need them, I just shrug and deal with the format we have.