I pretty much assume every smart home device capable of doing this is doing it already.

Nothing gets connected to wifi. If temporary connection is required it gets a random one time use guest network SSID and pass that I remove after. (Yes they can collect and send setup metadata during this temporary access period but sometimes it is a worthwhile tradeoff)