How is it not a reflection on the argument? If I say something is good advice but don't follow it, I clearly don't believe it myself in every respect. That seems highly relevant.

Again, that says something about you, not about the argument.

I say people should get some exercise every day. In practice, however, I only get to the gym once or twice a week. That means I'm a hypocrite (and a lazy one at that), but it doesn't mean people shouldn't get some exercise every day.

But it says you don't believe your own advice doesn't it? If you believed it, you'd exercise despite your laziness. Or perhaps it would be more accurate to say you believe a counterfactual more. I'm pretty confident about the effects of exercise, but we're talking here about hypothetical difficult questions. If even the person who supposedly knows the most about a particular argument isn't convinced by it, how can that not reflect poorly on the argument?

But even if I accepted that pointing out hypocrisy is purely argument by ad hominem, I don't see how it would be irrelevant. If someone believes something radically different from me and therefore works towards radically different outcomes, I consciously try to stay open minded in case they know something I don't. If someone isn't consistent in what they profess and what they work towards, I don't feel I owe them that sort of hearing. At best, their ideas are good but they don't understand them well enough to represent them. Would you really call that unfair?