Well, there are plenty of studies that show high contrast makes for readability issues. For example, I get faint after-images that flick around the page when reading black text on white. It's very distracting.

I get your point about different perspectives, but for web pages we want everyone to read, we should be designing for the lowest common denominator.