I just finished reading Godel's Proof (a non-textbook account of the proof recommended by Hofstader in GEB) and it was mind-blowing. I really recommend that and David Foster Wallace's Beyond Infinity. Beyond Infinity is great history of set theory and written in an exciting and fun style.

I came across the ideas the opposite way...I loved math as child and as engineer but I always felt like systems of rationalization had to be limited in some way. I think I started feeling this way when I'd get into debates with other students in school about arguments and we'd have equally rational but contradictory viewpoints.

Fast forward a few years later and I learn some genius had actually proved that we can't answer everything with these systems...or that there isn't a system that can say it all.