As someone who used to give away hundreds when he was on the average wage, no, it didn't really cost me life experiences. When I was on less than the average wage, I never had hundreds to give away, but on the average wage (with no kids), a few hundred didn't make much difference.

Few people giving to charity actually sacrifice their own life experiences to do so. As a result, it seems silly to dismiss philanthropic billionaires. We want more of those, not fewer.