Using the term literally is usually the irritating bit. Like responding with "appeal to authority" versus "Einstein wasn't an expert on xyz, though".