Re:How to share data with a statistician
by sanskritabelt on Wednesday November 27, 2013 @02:15AM
(#6806928)
In my career, and maybe this is just the kinds of problems I've been presented with, I've found that assuming normality is, by and large, like drinking tequila. It's fun at the time, but I end up regretting it in the morning.
EDIT: its this kind of experience that make me reach for things like nonparametric tests and the bootstrap.
How does sanskritabelt solve the problem, I wonder? Set an arbitrary number of volunteers for each group to say "this is the point we can stop collecting data", assume normalcy of ratings, take the means and use some test to see if the difference is significant or not?
Really the problem of the problem is defining what is meant by 'better'. Is 'better' a higher score? (Which scoring function do you use -- mean, median, most 5-stars, least 1-stars, something else that takes into account the number of ratings per rating bucket?) Does it have to be higher by a certain amount (how meaningful are deltas in your scoring function)? Once it's defined what is meant by "A is better than B", then you can go about the business of computing the likelihood of your data given "A is better than B" to fulfill the RHS of Bayes' theorem.