Generally speaking, if it wasn't in my ancestors' environment, I avoid it. Otherwise, approach with caution.

Nature can be dangerous, too, but at least it's likely that someone in the past has dealt with it and lived to reproduce.

With new synthetic substances, you're essentially doing alpha-testing with your irreplaceable hardware/software system...

Pollen in particular is weird because simply being outside should expose you to some level of pollen, compared to foods like melons or peanuts that may or may not be part of your diet. I'm mildly allergic to dust mites and they're literally everywhere.

Right. I'm currently undergoing allergy immunotherapy, so back in October I had a battery of allergy tests.

Turns out I'm allergic to some trees, some grasses, pollen, mold, dust mites, and a bunch of other things. And yet my childhood from a very young age was filled with playing outside in the yard and and in the woods.

Also my family had (indoor-only) cats since before I was born. My parents were allergic to them, and my sister and I are also allergic. (Wanting to get a cat is what prompted me to start the allergy shots.) So I've been exposed to cat dander since I was an infant. And yet: allergic. Some things are just hereditary, I guess.

I'm sure there are some cases where not being exposed to something when young can cause you to develop allergies, but that's definitely not the only cause. Seems like there is definitely a genetic component that can moot any amount of childhood exposure.