You can call holding multiple places on the field every single play. If they did that, the game would be ridiculous and nobody would want to watch or play. So they exercise discretion and only apply the penalty when the holding affected the play.