It's tax avoidance because if you hadn't gifted that $100 to charity then your taxable income would have been $100 higher and you would have paid higher taxes. You "avoided" paying tax on the amount that you gifted to charity.

By that definition, taking a lower-paying job is "tax avoidance" in exactly the same way.

Not really. Taking a lower-paying job not only avoids taxes, it lowers your income (both pre-tax and after-tax).

With charitable giving you can avoid taxes but still retain exactly the same after-tax income (assuming that you were going to gift to charity regardless of whether you got deduction or not). That is, you can gift $100 to charity, deduct the gift, and so avoid (say) $30 in tax. In this case the gift to charity effectively results in you having $100 less than before. Or, for same net result to you (minus $100), you could gift (after-tax) dollars to charity, say $70, and pay the $30 in taxes that you avoided in previous scenario. Only in second case you ended up giving only $70 to charity, because $30 goes to government. In first case you gave all of the $100 outlay to charity, avoided the $30 tax.