This was really the point that I was getting at. It is ultimately cheaper to put good chargers at every restaurant and bathroom stop than it is to put battery swap stations (with inventory) at enough locations.
The major one being your first paragraph. Charging tech is getting better every day, and we are already at the point where 15 minutes can get you a good chunk of charge with current batteries in some cars. I think things will change (no more "gas stations" (or places solely devoted to providing fuel), and a lot more charging spots at destinations that are worth spending 30 minutes at), the bit of charging time every few hours will begin to become a non-issue, and the whole complicated logistics nightmare of managing a fleet of physical battery packs at various depots across the country will go the way of the flying car. That thing that everyone thinks they wanted but it was just never practical to have.
Another thing to consider, it is much more likely that we will see 400mi range electrics sooner and/or easier than we could roll out a large network of battery swap stations. I think the concept of battery swap is solid. Most batteries have BMS systems that prevent over/under charging and any physical damage can be detected. Plus the battery would likely be owned by a third party, so it would not be a big deal if you got stuck with a bad battery, you would just swap it out for another one. The main issue is logistics. Much harder to build a network of battery swap stations then it is to put some fast chargers in parking lots. And if we can get the range up to 400 miles, then a 600 mile trip could be done with 20-30 minutes of super charging.
[1] Assuming a ~300 mile ICE range, you need to fill up at the start, at the middle, and at the end (although one could wait till the next morning, but I think it should still count). Whereas electric, you start full (because you plugged in overnight prior to departure), and you end full (assuming you can plug in overnight at your destination). These are rough assumptions, but I think 2 fuel stops is a good approximation, but many would need 3 (including my own ICE car, which gets ~250 miles per tank).