I recall a Reddit thread on this where many people chimed in with their experiences, which seemed to have a distinctly climate-driven element:

* Shoes off = don't track mud and dirt inside

* Shoes on = don't expose yourself to insects

And then depending on cultural background they would tend to keep one or the other habit after moving to a different climate norm. The shoe habits predate modern construction, so they may not matter like they used to.