I don't have the experience to know myself, but I imagine that there are various triggers of early mathematical derailment. It would be interesting to see a list of common causes.
Personally I find it hard to internalise canonical notation. Like f and F in probability theory, which is which again?
I have come to believe that the main trigger by far is the attitude of society. Of parents, family, friends, tv stars, heck even many (non math) teachers. "I wasn't good at math haha" is such a standard phrase to hear, and parents telling their kids that they don't need to worry if they "don't get it" as if it's some mystical topic that only a few gifted can unlock. Plus the uncool stigma attached to "math nerds", folks who simply have an open mind to try to "get it", turns out that it isn't actually that hard. At least when talking high school math or some basic college classes.
Probability theory's notation isn't very canonized. The typical usage, f for PDF and F for CDF, is easy to remember from the calculus notation of uppercase being an integral of the lowercase.
I ended up copying it by hand along with every exam and test notes over my entire degree into one little moleskine notebook. its a god send any time I have to remember how to do something or learn something new.
Students with math and physics backgrounds are fine with Greek letters and other mathematical symbols, but the biologists in the class are mystified. They also get terribly confused when I reuse symbols for different purposes.
What I've discovered is that the students who have trouble with mathematical notation and reasoning got derailed when a teacher, in an early grade, said "let x be the unknown". That is a phrase that never comes up in other contexts, and I think it throws them off track. Many find it difficult to get back on-track later, so they memorize and sleep-walk their way through other mathematics classes until the system no longer insists that they take them. A shame, really.