My understanding is that the gesture began as a kind of pat-down to show that neither party was hiding weapons in their sleeves (think "roman" handshake -- clasped at the forearms), and gradually evolved to what we have now.

Similarly, I heard that clinking glasses originally came from the host pouring some of their drink into your cup (and vice versa), thereby mixing the drinks and proving that neither cup was poisoned. Eventually this evolved into the symbolic tap we do now.