I don't find it to be a curse to be honest. There are things I am hyper-sensitive about (color) and things I am not (smells) and I actively seek out the things I care about and notice because they bring me joy.

And to be honest, the things I am hyper-sensitive about don't seem to really require more brain power. To take a very contrived example, when some company adjusts their logo's color by a smidge, it takes other people a while before they notice a change but I notice it absolutely instantly. To use electronics as an comparison, it's like my brain has a silicon wired to notice color instead of relying on 'general purpose computing' like a lot of other people's brains and so the cost of differentiating color is practically free.

And interestingly, because I am sensitive to color, much of my interests revolve around subjects where color matters (fashion, art, design). I like to meet other people who are equally sensitive to color because they will notice the things that I notice.