Unlike clothing, bags touch a lot more things, and being textile means that contaminants are easily absorbed and more difficult to remove. That's the argument I've heard, in any case.