Do they understand what is happening to the label that they identify with? Do they seek to make a clear statement about what they believe the alt right stands for?

If they let some definition take hold that includes racism, then in the future when they object to saying that the alt right is racist, they aren't standing up for what they believe in anymore, they are providing cover for the racists that stole the label from them.

Racists are not the people who are defining the term - it's the left and the media.

Who changes the meaning is totally irrelevant to it becoming cover. It's not fair to the people that aren't racist, but it's irrelevant to whether they end up providing cover for racists.

I would expect libertarians to be considered "alt right" by the average person.