That’s definitely true for me at least for chili peppers. In fact I think my reaction has worsened over time even after decreasing exposure to them. Such a pity—makes food taste so good.

Another example is Fijian 'kava' (a root that is crushed and dried into a powder, then filtered into water and consumed as a drink). It has a reverse tolerance: those who consume it more frequently typically get more affected by it, and those who've had little before may not feel any effect.

Always a delight to see kava being discussed in unexpected places. Say bula at r/kava!

Sometimes they get worse, too, like with cat allergies.

No one is talking about alergies, just life. Relax. Get dirty.