It's because you still have the one down left to try to get a first down, but instead you're giving up on the offensive drive by punting.

Also, I wouldn't say that it's completely synonymous with "giving up". It's more like "giving up for now".