Replying to myself, after a bit of research, it seems like "wearing crinoline" is actually a marker for being more concerned with fashion (e.g. hoop skirts). Perhaps a modern day equivalent would be saying that someone looks down on people who "wear lumberjack shirts with their hair high and tight" (e.g. hipsters).

Example: http://www.wondersandmarvels.com/2012/08/death-by-crinoline....