The problem is that the "graphics part" is often the part that knows how to deal with fonts, which you sort of need for layout.