Who you follow is a reading list for people who share your interests. Polluting it with people whom you follow just as a hat tip is rude. Also, following uninteresting people makes your stream unusable. In my book, keeping a high signal to noise ratio is the epitome of politeness. I guess that we don't use Twitter for the same reasons...
The rude part is that someone took the microsecond required to follow you and now they have to use a tool to find their unfollowers (you), unfollow you, and follow someone else.