Not at all, anyone can "join" the network, but nobody can decrypt the traffic without the key.
Zigbee has built-in, easy, encryption. You simply set your router to allow unsecured joins temporarily whenever you get a new device, which then learns the key.
When the network is set to allow only secure joins only devices pre-configured with the key can see anything other than encrypted frames.
As these devices become more popular, we'll start seeing more Zigbees and Z-wave devices become commodity hardware.