I see your point. One thing I've seen people do is to keep their debit card in the cover of their phone. Which is a great hack :-) You can just put your phone on the card reader.
In Romania, a bank started issuing watches that function as contactless credit cards last year.
https://embeddedsecuritynews.com/2017/06/romanian-lender-int...
Then add that you don't need to carry a wallet if you have phone payment. Throw your phone in a sleeve (or use a watch!) and go running, stop for your smoothie on the way home.
Finally, since it's a phone I can call it when it's missing or my watch will tell me if it's in the Uber I just got out of. Even works when intoxicated!