This is the key point. There are people who aren't trying to support a family as a sole wage earner.[1] If you raise the minimum wage, those jobs go away.

[1]https://www.bls.gov/opub/reports/minimum-wage/2017/home.htm