As someone who started their career in an office environment, then went to grunt work, and now back in an office and a remote WFH job. I don't think it would be a struggle. Would it be different? Of course... physical labor is the complete opposite of office work. Would it be harder than sitting in a chair pressing buttons?... sure. But it wouldn't be a struggle. It would be more of a struggle working with people that don't understand what context is or what nuances are.

It's not the work or the culture, it's the people. The type that would call you a vagina for wanting water on a hot day to avoid dehydration. Or the boss that will tell you "you think too much" when you come to them with an idea that increases productivity. But ya, the work or culture would be a piece of cake to navigate. For me, often time it's the idiots you have to work with that usually make a place a shit place to work.