It's not so much about the exceptions as it is about the domain the law applies in. Someone else mentioned Newtons law of gravity: In the very large domain of "stuff we can see and touch", Newtons law works perfectly. Even on the scale of planets, it's so close that it took centuries to detect that it was off. On the scale of atoms, it breaks down, so we can define the domain of Newtons law as the interval "bigger than molecules to the solar system" in which it can reliably be applied.

The bigger and better defined the domain is, the more useful the law. If the domain that you can reliably apply the law to is ill defined or has many unpredictable exceptions, it's probably not a very good law.