I'd encourage you to give others the benefit of the doubt. Without other evidence, that "perception of being passive-agressive" is only that, a perception.