News for hackers. Stuff that matters.
Re:Google Now vs. Siri vs. Cortana – The Great Knowledge Box... by sytelus on Thursday October 09, 2014 @07:24PM (#8435511)
If you want to do this kind of test with more discipline, you would give out random phones to randomly selected N people who have never used any of these services before. Then log each query they do for a week or two. Afterwards you can run same queries through all 3 services, generate results and do human judgement on mechanical turk about which one is the best. The science of measuring these stuff is complex and I've omitted many complexities here, for example, you want to do multiple judgement for same query, you need someway to measure judgement quality itself, you need great judgement guidelines that covers edge cases etc. Usually in organizations that work with big data, you will find competent measurement team building tools for these kind of measurements with years of investment.

There are no comments on this comment yet.