By the way, this might not be true in the US, but in other places there's usually some sort of shared transit. Even if it's not a city bus or a ride-sharing network, there's usually some person with an unofficial "taxi" that drives others if you give them a call, or someone with a passenger van that makes a weekly trip to a city for a few regulars.
Old and poor people everywhere can't drive cars, so these solutions are widespread, you just need to ask around (calling the local shops usually does the trick).
I actually have moral objections to both Lyft and Uber. Other than that I only take vacations in places where public transit is an option. For more rural vacations, I usually just ride my bike there. Plenty of fun places to bike to from where I live (Portland, OR).
Otherwise, yes, someone else does the driving. My partner and I go out of town by car once or twice a year and she drives.
It's a tradeoff, and obviously if you want to go places where there's no public transport options at all on a regular basis, it's probably a good idea to at least have a license, but for my part that's simply been enough of a reason to bother getting a license.
I turned 16 about 20 years ago and had 0 interest in a driver's license. I lived in an extremely rural area. My parents (rightly) said that I'd need to get a job to pay for a car. I figured why bother since I'd be paying for a car to drive to the job where I'd make money to pay for the car.
Seemed like a complete waste of time and effort.
20 years later, I still have no plans to get a license but have changed my tune on work a bit.