I agree with you, but I was responding to the request to keep "all the history". My understanding is the ref count would be the same if you merge in all the history or keep the branches around, so in those cases it wouldn't make a difference. I could be wrong though -- I don't know enough about git internals to know if one is worse than the other.
This, unfortunately, is not a great thing to propose.
Many of git's operations on refs are basically O(n). You really don't want to accumulate an unbounded number of them over time.