Re:You Only Need to Test with 5 Users (2000)
by arathore on Monday July 15, 2019 @02:15PM
(#20443559)
The article talks specifically about usability testing - though it isn't in the title but it is in the first line of the article. I don't think pricing strategies or matching algorithms and such would fall under this domain.
Re:You Only Need to Test with 5 Users (2000)
by julienreszka on Monday July 15, 2019 @06:31AM
(#20439479)
you probably don't but you probably should
If you are interested in (for example) determining the optimal price and only 1/20 users buy something... you still probably need about 1,000 X (price points you want to test).
In that "test" you are basically trying to uncover the demand curve (or points on it). It's a statistical question.
Say you have a dating app, and you are trying a new matching algorithim. It will also take thousands of matches before you have the data to make determinations.
All that said, I totally agree with the author. I would just frame it differently.
The question you need to ask is "do I need statistics?" Statistics have become habitual, but much (most?) of the time, we don't need statistics.
If you want to learn if a user can write and publish a blog using your software or install a water filter under they sink... you don't need statistics. You need to know where most people get into trouble, and n=5 will work fine for that.
This is intuitive if we just think outside of the "testing" vocabular. You write a CV/essay/article. You ask 1-3 friends to read it and advise. You don't produce statistics.