The idea that no one should be allowed to work from home because some people can't has nothing to do with "fairness". It's the same mentality that says "you kids should have to suffer the same things I did when I was your age", despite the fact that conditions have changed.

I am genuinely sympathetic to people who work jobs that cannot be made remote, but want to work from home. I hope that they will be able to find a way to achieve that desire. But no one gains anything when they force other people to stop working from home just to prevent a "disparity" like that.

There have always been, and will always be, differences in the conditions that different jobs require. As some others have commented, some people have to work shifts; some people have to work outside in bad weather—yet we don't see a clamoring to make everyone work shifts, outside, in bad weather. This is just a very unusual time when some of those differences are changing in a very visible and very widespread way.