I remember reading an article (wish I had the link) from a seamstress that talked about this.

It mostly came down to examining the stiches and seams for quality, which is what I always do now when buying clothes. Look for unraveling, loose or missed stitches, not-straight stitching, etc. Pull the seam apart lightly and see how it holds up.

Also things like zipper quality (YKK is a top brand, and would be stamped on the underside), or how well stripes are lined up. Do the button holes look well stitched?