I think their point was that the person has to be at your house. E.g. they can't systematically and remotely try the door handle of homes halfway around the world.