A good example of extending TIFF with tags is Pixar's PhotoRealistic RenderMan. It uses a TIFF flavor as a base for its mip-mapped textures. Such a TIFF contains a pyramid of layers of the image at different pre-downsampled resolutions and a bunch of custom tags, the renderer interprets when it reads such a file.
However, I can use 'Open As...' in Photoshop and open this as a TIFF. I will simply get the highest resolution layer from the file. That's a simple and battle-proven example of why TIFF is indeed a great answer to the problem.
.PDF .SVG .DXF/.DWG
All of these are open standards or basically open standards which are already widely implemented and allow the use cases mentioned (vector and raster combinations and layers) and also many of the ones you have thrown in.
Of course the real problem is that they are far too complex for most people's needs so why bother.
People really don't really often need to share the 'layers' of images combined with vector data (usually you use the layers yourself then just share the final product - a flattened JPG or similar), however if they do the above formats are there for them.