Surprisingly - as stated by others - memory is crucial to math and problem solving. Most problem solving is just memory on a higher domain anyway. Break down a problem with first principles (hard part), and solve with a mental model gained from memory.

Euler - probably the greatest mathematician of all time - had an amazing memory. He memorized the entire Aeneid and could recite any line at will.