Some places have church tax based on your religion, such as Germany. I've heard varying claims from locals about the difficulty of leaving organised religion, though for me as an Auslander it was as easy as just saying "none" in the right box when setting up my tax ID.

I am aware of Germany but that's an exception. In no other (developed) country are your taxes tied to your faith.