I went to Cornell College, which schedules classes on a "block plan", one class at a time for a month each. My first semester, I took four consecutive courses in calculus. By the fourth one, I was extremely burned out on math and barely squeaked by.

I ended up taking a couple more math classes before graduating (discrete math and linear algebra), have continued to study math on my own, and have recently been contemplating a master's degree in math, just because I want to learn more.

All of that is to say, I'm not sure that the math brick wall is constant, but perhaps sometimes you need to take a break to allow your mind to digest what you've learned so far.