Each input of the transaction requires a signature, which makes the transaction larger and therefore increases the required fee. Here is a random example from a recent block of such a transaction with many inputs, it pays about $52 in fees:
https://blockchain.info/tx/ef29f862f3ceee10ed3e1f04585e18cc2...
So it is more accurate to think of the cost as a fee per byte instead of a fee per transaction. And there's a minimum amount of bytes that's needed to consume any hitherto unspent transaction output.
Interestingly, if bitcoins were to support Schnorr signatures, then the signatures could be combined and then your idea would work. Though this would not be able to salvage the existing unspent transaction outputs that are already secured with ECDSA signatures.
Obviously that's pointless at the current fee level (<$1) but if the fee grows high enough, or if there's a way to reach these wallet holders en-masse and convince them of the value this would provide to the network (or get them to short btc in advance of a large sell-off of the gathered coins). Notwithstanding the legality/morality of any of this, it's not my field.