The biggest distractions in the office are the coworkers. Random hallway conversations, loud talkers in neighboring cubes/offices, people walking up to your desk with a question that should've been an email/chat (but really they are too lazy to check the ticket system/wiki/doc store for the answer). The level of distraction varies with each type of job, but for many quiet focus time is important. Working from home provides that quiet focus time.
20 years ago my boss ordered me to work from home one day when I had an urgent task to get done. Most of the time though I was better off in the office - while I couldn't get as much high concentration work done, I was doing more useful work because I was in sync with the office.
The above assumes that a significant number of people are in the office everyday. A pure work from home environment works because people figure out how to use other means to communicate important things. However communication is much harder, and thus takes more effort. As a shy person I would be much better off in the office all the time as I don't put enough effort into communication.
There's way more distractions at home and not everyone can handle it. It's just wild what people find appropriate to do during meetings that they're supposed to be paying attention to. In office yeah maybe someone would read emails but you wouldn't be fully immersed in doing something else like cooking a full meal.
It's hard to say how much my coworkers actually work. But I strongly suspect that it's much less than in office. Yeah people would pop over to Reddit or whatever. But that's far less engaging then going anywhere you want or doing anything you want. Feel like for most people if you're stuck in the office you might as well do some work.