This problem exists in physical buttons too. My thermostat has buttons that can make the mechanical click without actually activating the button, and I've experienced them elsewhere. It is infuriating.
You are both correct. That’s one of the reasons I mentioned multi-sensory feedback. Every little bit helps. Now, sound can of course be annoying, but it’s up to the discretion (and taste) of the designer to make that call or not – as always, depending on the context.
So having clear visual feedback that a touch registered, and showing you where it registered, is huge.
I reserve a special curse for people who implement touch feedback separate from touch processing, though. I've used a couple different apps where it's possible to create a visual 'touch' that doesn't actually trigger a click on the button, and that's infuriating.