Absolutely. Being able to instantly recall concepts, definitions, syntax, etc. is incredibly empowering when solving problems, regardless of the subject (mathematics, programming, writing, etc.).

Nobody would suggest that you try to learn a new language without memorizing words, phrases, idioms, etc. Communication is the foundation of any field of interest, so why do we think we can get away with "learning" them without memorizing the sub-language of the field?

Just as memorization is only part of process of learning a new language, so it is for learning in other areas.

If all you have is some rote memorization about mathmatics you can use those memorized items to study in your own head. So while you shower or drive you can think about the math you've memorized and develope a deeper understanding.

As you say, memorization can be a useful part of a deeper understanding.

Learning without thought is labor lost, Confucius said.

At any rate, symbolic representations of knowledge should be committed to memory (or pad and pencil) at the time of understanding (Hebb's rule), or else your "memory" will be isomorphic to random symbols and therefore inaccessible to future thoughts.

That said, some of the greatest mathematicians (e.g., Gauss, von Neumann, Dieudonné) had phenomenal memory, so I suppose rote memorization isn't a terrible hack to try to approximate the advantages this kind of gift confers, but only in a pinch, since these mathematicians were keeping much more than definitions and statements of theorems in memory.