How do you explain the success of American Giant? They use US grown cotton, make all their clothes in the US. Granted, you're not going to be able to buy a $5 t-shirt from them, but a $40 t-shirt that doesn't rely on crappy labor practices seems like something people have gotten behind.
Same with New Balance. Their shoes compete with Nike made in Asia, and are MIUSA.
We've just become addicted to cheap crap shoveled into Walmart and Amazon.
But even if you fixed that the sub component infrastructure isn’t there. The raw fabrics are simply not available here so even if you did pay labor of 160/$ a day you would still not be able to get raw materials