These things also have a habit of growing arms and legs - so another issue is the number of people with the key. I can almost guarantee that at some point the number of people with the key will grow (police forces, councils...)

Yet another issue is oversight of those with the key (who watches the watchers)