Not entirely relevant but note that if a power switch does not have a mechanical difference between the off and on state then there must be an active component that at the very least must monitor the power button to detect presses. I don't really know, but it seems likely that in a smart phone the SOC is used to monitor the power button and it is therefore only ever in a very low power state (sans battery removal). That doesn't of course mean that any radio components are receiving power.

I think you could do that in a passive way as well, where the act of pressing the button powers up a circuit that holds the power on.

low power devices now exist that can live of small traces of power. Even with the battery removed, how can you be sure it's still not listening?...

This is why you should never take phones into any kind of sensitive meeting or at least put it into a faraday bag.