As a developer (of the cash register software, which wasn't at fault in this case), I was kinda amused by the whole thing. But I can assure you, the higher-up business people weren't amused at all.
That bit us in the A$$ with double payments a few times.
0- https://hc.apache.org/httpcomponents-client-ga/httpclient/ap...
You should embed a transaction ID to ensure idempotence of your API call. Then you can make the request 1, 2 or 3 times without any risk. The receiving end will realize the transaction ID already transitioned state and won't cause multiple payments to occur.
Most of them came from.a gateway called CyberSource.
Also, lots of customers have weird banks where they see both an authorization (I.e. hold) and the actual capture, and they mistake it for two captures. Usually it helps to educate them about what is an authorization.
Most banks don't even bother showing the customer authorizations of any kind.
The race condition resulted in an error syncing between two datacenters, so when it happens it can be dealt with manually.
Also, to be clear, it did not affect customer waiting time as the delay was in a webhook which we used as a backup to the main payment process.
Oh yes you can, and do, often, in payment processing code. I've done it (with a simple sleep for x seconds I think), and I can assure you the higher ups couldn't have been happier.