How is that fiddling? Just put the phone to the terminal, that's it. Sometimes press finger to the fingerprint sensor.

For card, you need to get the wallet, get the card from the wallet, and then put it back.

Some people don't use wallet or put it close, because I only need wallet to pay or use loyalty card, and I can do that with my phone.