I had the same annoyance as you, switching back and forth wasn't pleasant, and I'd forget. I think part of the reason is that switching between 0% <-> 100% saturation doesn't give room to adapt. It feels like a bandwidth of information is missing. But, if the colorspace is barely noticeable, your brain will fill in the colors for you with the information it gets... and the high-saturation default color-set would start to appear abrading and unnatural.