But if you can work remotely from home/desk, so can people who are willing to work for 10% of your salary. In person means "Im on the radar, im there to consider, and there to help". Someone over a chat client doesnt have the same presence.
Now, this might/probably change with the advent of AR and VR workspaces.
That's just capitalism. Choose a profession where you're not so easily replaced by someone who will take 10% of your salary.
If you're so unskilled as to be that replaceable, there's no reason to feel comfortable in a non-remote job, either.
Just take a software engineer for example. LOC is fallible in too many ways to take seriously. Number of function points coded? Number of bugs fixed? They are all very fallible.
How about mentoring? How about reviewing code? How about documentation? How about professional development?
Depending on the situation, reading a technical article can weigh higher than a support issue. How do you develop the weighting for the different factors? Can it be applied across organizations?
To confound it further, people tend to take on work that doesn't fit their title. The smaller the organization, the larger the diversity of responsibilities.
Then there are soft factors like knowing which issues to address immediately in order to keep them from percolating up to your manager, working well within a team, communicating effectively, etc.
My opinion is that we are unlikely to see objective, effective measurements of productivity for the Tech Industry anytime within my employment life. It's not a failure. The number of factors involved, not to mention the ability to gather the measurements, is staggeringly complex. After all, we're not factory line workers producing pre-defined widgets.
Depending when you were out at the pool, I can imagine you got a few instant messages from your boss that you didn't reply to.
Now I'm back in an office doing 9-5, and everything is a chore. My stress levels are significantly higher than previously, etc...
With tools like group video calling, conference calls, email, Slack/chat, etc... It's incredibly easy to stay "close" with your team while being remote.