That's not a correct inference. It might be that all textiles produced make it to people's homes, but people also throw away lots of old clothes. Another reason might be that some textiles are not made into clothes, and are used only briefly for the purpose they were created before being tossed.