I don't understand what your complaint is here. You do something that is considered rude in both of our cultures (as evidenced by how you say people respond to you), but you appear to be aggrieved in some way that this is the case.