No‚ it's not. They're not attacking the validity of your statement by criticizing your tone, they are criticizing the effectiveness of your communication. And tone is completely relevant in that context.