But, if you store a SSN as say 123-45-6789 your using at least 11 bytes to store a 4 byte number. Use it as an index and you waste that space again, store it in a cache and you have less space for useful information, compare it with another SSN and it's slower etc. Now individually it's probably irrelevant but start doing this with several fields on a large application and you can noticeably slow things down even if the extra disk space seems cheap.
PS: 95% of the time it's probably the safest option, but it's still something you should consider based on how that data fit's in with the rest of your application.
Just a couple of problems: some day in the future the field could need another character beyond 0-9. In Spain National Identity Numbers were once just numbers, but they changed and added a letter for validation. If you had stored this as a number you would need to change data types, and suffer a massive application update.
Another problem: if you order a number field you get something like 1, 5, 100, 2000. But if you order a text field you would get 1, 100, 2000, 5. It's not nice to get the "number" order when you are expecting a "text" order.
But the issue is with storing them in numeric datatypes. They aren't guaranteed to be unique forever (and indeed, are alphanumeric in many jurisdictions) and the leading zero problem is a killer when you go to reconstitute them, both for performance and for logic reasons. Same goes for US ZipCodes.
This is why I have to have a very strong technical reason to make a non-math column numeric, especially with externally set data (like SSNs, SINs, Account codes, etc.) The people who set them could just start adding letters or symbols...and this isn't rare.
Plus it gets worse because people inevitably end up writing queries that parse the number with dashes, brackets, or other special delimiters. Those queries are CPU-intensive to do on the fly, and with databases like Oracle, DB2, and SQL Server charging by the CPU core, it becomes an expensive proposition as well.
(I don't keep to this rule for surrogate keys which will be int/bigint auto incrementing, even though I won't be doing any math on the values themselves)