I do that too all the time. A downside though is that you risk ripping the material, especially with knitted shirts. I've done that only a couple of times over the years, so it is not a big loss.