> Pet peeve: white is itself a color experience. It's not colorless.
The way that I thought about color before being educated in the properties of light was to think of white as colorless. It's the thing that's easiest to add color to in a subtractive color model (e.g. using crayons paints as a child). It doesn't represent my science-based understanding, but I wouldn't be surprised to observe someone looking at something white and saying "Huh, it needs some color".
How much of this is just an artifact of the fact that all our writing surfaces tend to be shades of white?
To expand on your point, retinas talk a limited number of overlapping spectral responses. That's why we can have an essentially identical perception of spectral yellow and a mix of red and green light.
To temper your point just a bit, even though yellow light and red+green light look the same when viewed directly, you can still tell the difference. For instance, in the way the light reflects off different materials, or by chromatic aberration with corrective lenses. This is why R+G+B LED lighting is not good enough for illumination and why some better bulbs instead use UV LED + phosphors.
Pet peeve: white is itself a color experience. It's not colorless.
The infinite basis that forms the spectrum is different from the basis of three cones (receptive to bands of long, medium, and short wavelengths respectively), which itself is different from the basis formed by color opponencies (white/black, red/green, blue/yellow). Since the projections that go from one of these spaces to the next are not 1:1, to speak precisely we should not identify a spectrum with a color experience. (And, of course, because of other complications that this skips over like lateral inhibition and habituation.)
Physics talks about spectra and wavelengths. Retinas talk certain bands of wavelengths. Only brains talk color. The spectra that usually make brains experience white can be separated into spectra that make brains experience other color experiences.