USB-C has two physical channels, and the act of having to decide that a channel needs to be used for usb or hdmi or displayport is a complication that hubs were never well specified to handle. Hence there not being many such hubs.

USB4's tunneling means that the connector does not have to enter an alt-mode. Even when transmitting displayport or hdmi, the hub or hubs and host can remain using usb4. As well as making hubs suddenly make sense (after being an unspecified ball of confusion with alt modes), this allows bandwidth sharing that was previously impossible.

In short, that these other specs had packets is irrelevant, because those solutions werent really compatible with hub-like devices. There needed to be a common mode of interoperation, which hadnt existed before usb4.

>I think both Displayport and HDMI have official specs that work over the USB-C connector.

You've answered your own question here. "over the USB-C connector" meaning direct connections between two devices. The moment you put a hub inbetween it won't work.

USB C Port -HUB-> DP Port works, USB C Port -HUB-> 4x USB C port -USB-C-to-Display-Port-adapter-> DP Port does not.

It's adding a small percent overhead in exchange for making it routable over a hub. Without the extra header it requires dedicated lanes.