I tend to keep my shoes on in someone else's house, unless they are a relative or friend, or unless they say something along the lines of "make yourself at home." Although I don't wear shoes in my own home, I don't think it would be rude if someone else were wearing shoes in my house, maybe just weird. Another determinant is carpet vs hardwood floors: I'm more likely to wear shoes in a house with a lot of hardwood flooring compared to a house with carpeted flooring.

I can see why Americans would be viewed as tending to wear shoes inside, since I've seen a much stricter adherence to not wearing shoes inside in Japan (to the point of having separate footwear for bathrooms).