Is the best match dot product (or other distance metric used) below some level? What the good level is would have to be examined for your particular use case, maybe make a histogram of the best (or several best) dot products from many good searches and pick some quantile.