According to the BLS, workers earning the federal minimum wage or less represented only 1.4 percent of all hourly paid workers in 2021. Taking into account all workers (not just those who are hourly paid), this small fraction would shrink even further, so I wouldn't expect them to have much affect on the overall unemployment rate.