Re:What the SATs Taught Us about Finding the Perfect Fit
by avs733 on Thursday December 14, 2017 @04:06PM
(#15927045)
>The Rasch Model is a specifically simplified case of item response theory, but I'd argue that it may not be the best one for stitch fix.
Hello fellow IRT nerd. I was going to come post something similar! I love yours but I hope you don't mind if I flesh out a few details.
I think the biggest takeaway is that one of the underlying assumptions of Rasch is a set of observations of the phenomenon that collectively work to make an estimate of a uni dimensional trait. In this case both of those assumptions are germane. For starters, I would be really concerned about the assumption of unidimensionality. I definitely find variances in fit in more than one dimension for clothes.
I totally agree that ignoring discrimination (Rasch sets the discrimination of each item to an arbitrary value, 1) is likely problematic here. Rasch constrains discrimination because of the idea that the test (i.e., all the questions) discriminates (to be clear, discriminate == sort). If I was developing this model I would want to have that discrimination parameter. It would be informative as to what clothes are more effective at bridging different sized people (i.e., how flattering they are to differing body types). It would also get me away from the internal assumption that the model basically necessitates multiple 'observations' (i.e., pieces of clothing) to make an effective estimate of fit.
Similarly they might also make use of the 'guessing'. That parameter is effectively one to track randomness in a model. Maybe you have a piece of clothing with a weird shape or a different relationship sizes in the cut than others.
All of this model also ignores other variances between people and between brands or items of clothing. You can't really introduce these with strict IRT, but there are more generalized forms of IRT that can deal with multiple dimensions. If you want to get into predictor variables you can do things like multilevel logistic modeling (also known as multilevel measurement modeling) which are becoming more widely used in education. Those tolerate nesting of data which can be very useful. Maybe banana republic clothes fit me better than they do lsiebert because of where the arm holes are even though we both wear a L from every brand. That is a nested effect/interaction...it is not specific to banana republic, but specific to me and that brand. You can't get that out of IRT or Rasch.
General description:
1 paramater IRT/Rasch estimates 'difficulty' varies, akin to general size assuming size is a single linear dimension
2 parameter IRT estimates difficulty and discrimination (i.e., how 'sharp' the logistic curve is)...maybe a piece of clothing that fits slightly different shapes and sizes beetter or worse.
3 parameter IRT estiamtes difficulty, discrimination, and guessing. helps identify a garment with a random influence...maybe people tolerate the item less well because its a known brand
Hello fellow IRT nerd. I was going to come post something similar! I love yours but I hope you don't mind if I flesh out a few details.
I think the biggest takeaway is that one of the underlying assumptions of Rasch is a set of observations of the phenomenon that collectively work to make an estimate of a uni dimensional trait. In this case both of those assumptions are germane. For starters, I would be really concerned about the assumption of unidimensionality. I definitely find variances in fit in more than one dimension for clothes.
I totally agree that ignoring discrimination (Rasch sets the discrimination of each item to an arbitrary value, 1) is likely problematic here. Rasch constrains discrimination because of the idea that the test (i.e., all the questions) discriminates (to be clear, discriminate == sort). If I was developing this model I would want to have that discrimination parameter. It would be informative as to what clothes are more effective at bridging different sized people (i.e., how flattering they are to differing body types). It would also get me away from the internal assumption that the model basically necessitates multiple 'observations' (i.e., pieces of clothing) to make an effective estimate of fit.
Similarly they might also make use of the 'guessing'. That parameter is effectively one to track randomness in a model. Maybe you have a piece of clothing with a weird shape or a different relationship sizes in the cut than others.
All of this model also ignores other variances between people and between brands or items of clothing. You can't really introduce these with strict IRT, but there are more generalized forms of IRT that can deal with multiple dimensions. If you want to get into predictor variables you can do things like multilevel logistic modeling (also known as multilevel measurement modeling) which are becoming more widely used in education. Those tolerate nesting of data which can be very useful. Maybe banana republic clothes fit me better than they do lsiebert because of where the arm holes are even though we both wear a L from every brand. That is a nested effect/interaction...it is not specific to banana republic, but specific to me and that brand. You can't get that out of IRT or Rasch.
General description: 1 paramater IRT/Rasch estimates 'difficulty' varies, akin to general size assuming size is a single linear dimension
2 parameter IRT estimates difficulty and discrimination (i.e., how 'sharp' the logistic curve is)...maybe a piece of clothing that fits slightly different shapes and sizes beetter or worse.
3 parameter IRT estiamtes difficulty, discrimination, and guessing. helps identify a garment with a random influence...maybe people tolerate the item less well because its a known brand