You never end up with a net plus by giving to charity; you are always giving away money. This is because the amount of tax you avoid is always less than what you gift. For example, you may gift $100, which ends up reducing your tax bill by, say, $30. If charitable gifts weren't tax-deductible the equivalent would have been paying $30 in taxes and having $70 left over to give to charity (for same $100 loss). The tax-advantaged status gives charitable gifts an extra bang-for-your-buck by giving you same net result (total outgo of $100) while allowing you to direct all $100 to the charity of your choice.

Also, notice that you're not somehow "getting something back" from other taxpayers in this scenario. You're just lowering your tax bill.