The key never leaves the smartcard: the card hardware will not allow it to be retrieved. Rather, your system has drivers which ask the card to do all of the signing operations which your system would normally do if it had access to a key.