Sure, but don't let the perfect be the enemy of the good, buying something with a bit of spandex or whatever for stretch/durability is vastly better than buying something that's pure polyester fleece. Just go for as high a percentage cotton/wool/linen wherever you can.