It could very well be true that disabled people are more offended by third parties policing language, especially if the expression was used in a neutral context. Because that can be very humiliating and shifts the topic to something many people want to overcome in their lives.