Your proposed solution looks to be a mix of changing the color space and the metric. Maybe we could use HSL/HSV with a metric that reduces the weight of some component. It could therefore consider less the fact that the three components are high/low, but focus on the "real" color side.
I think working with HSL values works better in principle. Just adjusting the lightness value can be used to arrive at a more sensible color description. The most common lightness values for color names hover around 50% (at least in the 3000+ dataset I compiled). One solution is to have a simple formula that takes in account how accurate the nearest color name match is and if it is too inaccurate, to find the color name for the color's HSL with the L adjusted to 50%. I guess you could call it staying true to the hue. I think that would make the color naming script even more useful though.
With several points for each label, It will give us the possibility of using a 3NN, for example, instead of a 1NN classifier. It should impact also on the results, but I'm not sure it will really improve the results.
I was wondering what to do with darker colors as they are less represented in terms of color names. So, when you pick a random darkish color the accuracy reduces because there are less names to choose from.
I was thinking perhaps the best solution would be in these cases to extrapolate a lighter variation of the picked color and display that color with a measure of its darkness (30% darker purple). Right now, if you pick a color like #291c36 you'll get dark jungle green when it's a dark purple. Any thoughts on tackling that problem?