Maybe it's a cultural difference but I have a hard time thinking that this is impolite. I'd have do the same and it wouldn't mean that your french is not perfect but only that I would like to ease your life.