Just keeping a car on a road with no traffic takes a fair amount of focus: for evidence, just see how many people don’t do it and plow into objects that are alongside the road.
Obviously, driving requires attention and is always one's main focus. Having driven various sizes of vehicles, it's quite clear that this amount of attention varies. For example, the sheer amount of work required to drive a 26 foot box truck gave me a healthy respect for truck drivers. On the original topic, I'd imagine a lot of RV drivers don't pull over or are actively hostile to being passed because they're so overwhelmed they don't want to pile on any more requirements. And while this means that they should be taking the opportunity to stop and recover, try telling someone who is drowning to stop thrashing.
Piloting a 2500 lb manual coupe barefoot in a non-wooded area on an empty road requires the lowest amount of work I've experienced. Driving that same car while having to continually gauge how hard the person in front of you is applying the brakes takes much more. Braking extra and creating more buffer room gives a slight reprieve, but at the ailing speeds ruins views tend to go on non-straight roads you'll inevitably catch right back up.
I spent several months living out of my car+tent in national parks/forests and often referred to them as Ruins Views. Breathtaking scenery in those national parks, that you can't focus on when you're stuck in a line of twenty cars riding your brakes and smelling theirs, because Big Bob doesn't want to get his tippy palace going too fast.
I do get the appeal, but lugging around so much tonnage seems like a recipe for a bad time. Now I'm at a different life stage where I've been doing a bunch of towing with an underpowered SUV, and I make it a point to get out of peoples' ways. Luckily most of that has been on multilane highways where it's easy to do so.