We rinse the bottles/cans whenever we are doing other dishes. We throw them into a bin that holds all deposit eligible containers. When the bin fills we take it with us on our next trip to the grocery store and recycle them.
It probably takes me about 5 minutes to feed them all in and it recovers about 3 dollars.
These days I get it in change and then give it to my daughter for her piggy bank. She's very young, but we're trying to introduce the concepts of saving early.
The general plastic recycling is considered too contaminated to be used for food.
Knowing this, I give my bottles to the people who collect the deposits. (I don't wish to do it myself.)
There's a psychological effect. For example, the grocery store charges 5 cents per bag. If I forget to bring bags with me to the store, it irks me that I have to pay 5 cents, even though the amount is trivial and completely non-substantive.
They're not necessarily vestigial. NY's been considering expanding bottle deposits to juice, energy drinks, coffee and tea sold in plastic or glass bottles (currently, it's mainly soda and water).
FWIW have you done the math on the actual money you get back, divided by the time spent managing an extra waste stream? Unless you're a heavy bottle user, I personally don't see how it makes sense to not just ignore the deposit and recycle via modern means.