There are bandwidth limitations. My 120hz 4K main monitor supports HDR, but a single displayport cable can't push data fast enough to use HDR at that resolution and frame rate. I can connect 2 cables between my monitor and 3090 GPU to enable HDR, but that would break gsync.

When the manufacturers get this sorted out, you'll likely still have to make a trade-off between latency (240hz), resolution (8K), HDR (bit depth), and your suggestion (more color channels)

You don’t need more color channels in all of your images to support more display primaries (at least, to a basic level). You just need GPU-type hardware built into the display which can convert each small area of an image made of pixels with colors represented in a trichromatic space to the appropriate signals sent to whatever physical arrangement of varied-primary pixels to reproduce the intended color (assuming a human viewer with typical color vision).

Probably most convenient would be to start using 16-bit floating point numbers for each of 3 color channels, so that would still increase bandwidth somewhat compared to current high-end displays which use 10 or 12 bits per channel.

It would eventually cool to have spectral representations of images, but that isn’t necessary for just expanding the gamut.

ProPhoto RGB colorspace has room for some non-physical colors, but it still only has three axes for primaries. It just has a range broad enough to include most visible colors as well as some that don't correspond to a sensory input you can get with your three color receptors.

With paints and pigments, it's pretty common to have more than three primary colors mixed in. But with more non-orthogonal axes, there's more overlap where you can achieve the same color multiple ways by combining the colors in different proportions. E.g. purple could be a single purple wavelength or a combination of blue and red.