An individual set of card details are worth too little to a fraudster to be worth taking the risk of tampering with phone lines or breaking into an exchange followed by ages of monitoring (tapping the wrong line? mine hasn't been used for phone conversations for weeks - before that it was broken for weeks before I noticed) - it's not cost effective, and so not much physical security is needed for an individual line, nor that much for an exchange.

If it can be obtained through automated exploits, the cost-benefit ratio changes drastically.