Actually, although number theory touches a lot of "conventional" crypto (some of the design rationale for AES, polynomial MACs), most of workhorse cryptography in normal applications is not especially number-theoretic, and has more to do with information theory and statistics.

The belief that number theory is essential for cryptography is due to its role in public-key cryptography. But even if you're comfortable with number theory, new applications of public-key cryptography are tremendously difficult to get right, and require subject-matter specific expertise.