Vocabulary is a strong signal of social status (higher education is strongly correlated to wealth or social rank). So care needs to be taken to avoid signalling you are "better than them" to others that rank you as an equal.

Do you get accused of being condescending?

Maybe find someone smart with good social skills whom you respect, and have them watch your interaction with others, and then have them give you honest feedback?