An ad hominem doesn't have to be irrelevant; it just means that the argument is about the person rather than the position. Attempting to dismiss an argument due to the person's credentials instead of their position is 100% an ad hominem.

But like contravariant said, that doesn't mean the argument is wrong—just that it's a weak argument.