Sort the list by confidence when displaying it to the user, and make it easy to remove members - the potential false positives are most likely to show up with the least confidence.

After that, the incidence of false positives would be very very small.

Plus, what really is the penalty of false positives? FB already surprises people (at times in a bad way) by sharing things in ways they don't expect it to.