Tesla: "prevent the breeding of the unfit by sterilization"

Who decides who is "unfit"? In the not too distant past, and possibly even today, that might include the non-white, non-straight, non-wealthy, etc. I find it incredibly scary that you could so easily agree that others should decide who gets to enact their human instincts. Let's face it, it would almost certainly be old, white, wealthy men that would make those decisions.