Sometimes emergencies will necessitate missing even core business hours. This is more a matter of having more responsibilities (family, kids, etc) than being remote though. But if an employee has these sorts of responsibilities, I'm guessing they are more likely to take up remote work.
There are positives and negatives to hiring remote workers, and I think there are several ways to compensate for the negatives. Obviously since I work remotely I think the positives outweigh those negatives. But I also think placing so much emphasis on one of the negatives, that I rarely experience in practice, is extremely narrow sighted. But maybe your remote workers are bad at working remotely, maybe your company is bad at managing remote workers, or maybe there's confounding variables like the one I mentioned above.
However, like many remote workers I work in a distributed team across several time zones. So it's not just "he goes to the gym in the afternoon," it's more like one person leaves the office in one time zone in order to beat the traffic jams, while another person in another time zone has to pick his daughter up from school, another in a third time zone gets up really early but can't do night calls because of family life, and so on. Slightly wonky schedules actually give us more coverage in case of crises, and the only way we could ever line everybody up for the same work hours would be to fire half the team.
We manage it all pretty well by scheduling our meetings in advance, not being afraid to pick up the phone, taking ownership of our work, and having senior people in different zones who can help fix urgent problems.
(However I totally agree with the "need for trust" comment above. I worked locally before going remote in this job, and also in two of the three other remote jobs I had in the past.)
My preference would be to go to the office and be with my colleagues 2-3 days a week, and work from a home office the other days. We have some people who do that. But all the offices with direct colleagues are at least a 12-hour flight away from me, and I like where I live, so I have to just deal.
If I ever find myself leading a remote-first team I would make it a priority to have good strategies against isolation.
If you ever have to be a remote worker in the future, I'd highly recommend getting a co-working space as it really helps fulfill a ton of those social needs. Your employer might even pay for some (or all) of the costs associated.
I've tried co-working but once you add in your own chair and monitor (ergonomics) and eavesdropping-proof location (security) and taking the laptop home at night (security) and on top of all that having a commute again, it's kind of a pain.
I will probably eventually rent my own office (at my own expense) near where some friends work, but that's a pretty expensive way to guarantee social interactions.
I'd be curious what else you've come up with.
Here's how I stayed social. First, I used the time you would spend commuting or going out to lunch to get non-work chores done. I'd do laundry in the morning when I'd otherwise be in the car. I cut the grass or worked out during lunch, etc. I'd already be fixing dinner during everyone else's evening drive time.
That freed up my weekday evenings to be social, since I wasn't in after work commute-cook-chores hell.
I'd spend time at programming language groups; taking classes related to hobbies (brewing, a little woodworking); going out with friends to nice restaurants on Wednesdays when they weren't crowded; and doing some social volunteering. I got involved in running a couple of software conferences and joined a wine-and-movies social club.
I found—since I am a "gregarious introvert"—that not having to be around coworkers all day left me more energized to socialize with friends and family in the evenings.
And instead of a social circle dominated by coworkers, I have one that provides interactions with people from all walks of life, and have friends scattered around the globe to visit when I retire.
If you plan well and work at it, you can be more social working from home.
A co-working space gives you the option to have a social work space, but you get to choose each day (and even each hour) if that's what you want.
I think they other advantage is in a commute. You have many more options about where you want to be co-working than you do about where your company's location is. If your company is across town from you residence, and you have an hour commute you could find a co-working space that is closer to home.
I think we found your problem
It's not comfortable for me as an almost-every-day thing.
Agreed.
I honestly think that most people will not thrive in a fully remote, work-from-home environment.
That may well be true. But there's clearly a non-negligible group that swing very much the opposite way. What we're looking for is flexibility, not one-size-fits-all.
And yet people look at me like I have a third eye sometimes (including several posts up in this exact thread) just because I, like many people, don't like working from home.
The other members of the team seem happy with the arrangement. Maybe it would work better for my situation if I had a good home workspace available to me, too. There are other things I'm not happy with there, and I'm leaning toward thinking that my time there has just run its course.
There are more asynchronous ways to do things, and not always restricted to contractor-style arrangements.
Personally, I think it's unfortunate if the default way of working assumes such tight coupling: for me at least, it's inimical to achieving any kind of deep focus.
If you're not online and available during the expected business hours for your position then you're not really doing 100% of your job, because you're not able to assist your team members as is expected of your role.
Note that this applies to members of standard software engineering teams, and not necessarily to contractual work that involves delivering products on certain timetables but not necessarily close collaboration with a team.