I did when when still working in offices, as did almost all colleagues - wearing slippers, not without shoes. Not just because of the dirt but I also can't imagine wearing winter boots in a 23°C office.