I found abstract algebra to be really interesting. It kind of removes the concept of "numbers" and transitions it to operations on objects like sets or functions. You redefine operations like "addition" and "multiplication" to apply to these new objects, and see what happens. A whole bunch of interesting things fall out, many of them important to information encoding and error correction in practice.

For an example, see the concept of a Ring:

http://en.wikipedia.org/wiki/Ring_%28mathematics%29

Less directly related to math, I spent a joyous couple of hours back in college working through the Lambda Calculus wiki page, trying to every piece it. Once it "clicked" it became really fun to work through the examples.

http://en.wikipedia.org/wiki/Lambda_calculus

> Now I get scared whenever I see the "summation" symbol.

I'm not sure why, but I guess for me, removing the idea of "numbers" makes math suddenly fascinating. It's like dropping down a level to redefine what you thought was a well-defined concept (addition of numbers) as a generic operation for which numbers are simply one special case. Weird. :)