Regarding the sub-$5k car: it depends on your definition of "decent used car." Most people are unwilling to accept a used vehicle with a slightly broken interior or other minor issues. Just today, I was looking for comparative prices for a 2003 F-150 that I've got, and I found multiple in the $2-3k range in my area that are in good condition (running motor, A/C works, non-necessary repairs required). And that's a pickup truck: if you want a coupe or sedan, there are even more options below $3,000. You just need to be willing to accept some flaws: right now I've got several tabs open of good "A-to-B" cars with under 200k miles.
And at $250ish to rent a vehicle for the weekend it only takes a couple weekends to pay for the insurance, registration, and taxes (or is it just that much more expensive on the coasts? I'm in the midwest). There's a reason people own instead of rent, and it isn't because renting never occurred to them.
First of all, the days of getting a decent car for $5k in the US ended with the pandemic and the supply chain insanity. But even if you could do that, you also need to factor in:
* Gas
* Insurance
* Registration and taxes
* Maintenance
If you don't use a car as part of your commute or regular errands, it's likely cheaper to rent.
Honestly, the best compromise solution is usually to share a single car for a household. Not helpful if you're single, but if you have a partner, then you can usually get by fairly well with just one car and now your individual costs are cut in half.
Another way to balance the trade-offs is to own a car that's smaller and more cost efficient but doesn't cover all your needs. Then rent on the infrequent times when you're traveling, camping, bringing a bunch of kids to a party, hauling lumber, etc. and you need the extra storage or passenger space.