I am always going to be using a computer with normal colors, as are my cohorts. You can accommodate us or not, the burden is on us. But that means we are going to think your website looks bad and/or maybe isn't even worth our time if we have to muck around with tools to be able to use it.
Heres an example of what it does: http://imgur.com/a/giSxg and how bad it can look: http://imgur.com/wlWgC The top is normal, the rest are transformations I'll use (the 2nd image is the one I usually need).
I think the solution can only be to make sure semantically relevant differences in color (e.g. different positions in a chart) can be identified by people with dichromatic vision, e.g. by not using the popular disticton of light red vs. light green. Quickly viewing a design through one of those filter programs is no big tasks and should be part of basic accessibility checks.
This doesn't mean you can't use all kinds of colors when they are not relevant to understanding, [edit: removed direct reference to deleted parent content]
The mapping could work like high dynamic range mappings, trying to take into account local image features but this would probably make everything look really garish :(
I agree that a switch/mode in those color-modifying lense programs could be to remap color channels so a color blind person trying to read a specific chart or website could temporarily remap colors in order to better tell apart the colors used.
edit: disclaimer: I don't know about anybody with limited color vision personally so I just try to be careful assuming the problem could be solved easily. If testing with actual people with this condition proves me wrong, great.
Aside from that, some of the design lessons and considerations that come from considering color-blind users tend to be good design principles in general.So - they may be good considerations to make anyways.