That wasn't my experience. I just went to a ticket desk and they not only gave me my deposit back but also the remaining balance on not one but two cards. Wasn't a hassle at all.

That was at the underground Victoria station ticket counter.