"At school Feynman approached mathematics in a highly unconventional way. Basically he enjoyed recreational mathematics from which he derived a large amount of pleasure. He studied a lot of mathematics in his own time including trigonometry, differential and integral calculus, and complex numbers long before he met these topics in his formal education. Realizing the importance of mathematical notation, he invented his own notation for sin, cos, tan, f (x) etc. which he thought was much better than the standard notation. However, when a friend asked him to explain a piece of mathematics, he suddenly realized that notation could not be a personal matter since one needed it to communicate."

The story is either in "Surely You're Joking, Mr. Feynman" or "What Do You Care What Other People Think?". I forget which stories are in which book. Both are great though.