but that takes time, which means depleting cache. As most domestic connections are limited by upload, its not like you can get that cache back.

Especially if you are at the edge of the the p2p chain. Your cache might be dependent on three other people refreshing their cache first.