"The difficulty is that people used to agree (and at varying times, sue each other) over which metaphors to use"

I think you hit the nail on the head. A UI common library should not provide metaphors but direct access to the renderdable elements, their layout routines, etc. A proper UI library would resemble a modern 3D game engine more by its interface.

The level where UI metaphors step in would be built on top of this basic substrate.

There are oh-so-many projects out there trying to use this pattern but for some reason they've not reached popularity.