Picking/banging a lock leaves forensic evidence that a duplicate key would not, even when performed by experts. Google "lockpicking forensics" for lots of information.

Anyhow I guess your point is still valid. Someone wanting to get in and out of your house without living a trace could use some other means not necessarily involving the door, though most of the time they would risk looking suspicious to potential witnesses.