Re:The Startup Founder Wealth Survey
by nostrademons on Tuesday August 18, 2015 @05:18PM
(#10082261)
Look up "voluntary response bias" in any statistics tutorial. The general lesson you'll learn: "voluntary response data is meaningless". To draw any sort of valid conclusion, you really need random sampling.
If you can't get random sampling (which is often hard when you're looking at something personally sensitive like income or net worth), the best you can do is to select your population through some means presumably uncorrelated with the quantities you're trying to measure, and then randomly sample within that. For example, if you were a bank or other financial institution, you'd have data on everyone's net worth, and you could randomly sample within the population of business owners there to draw useful statistics. There's still some sample bias (eg. perhaps the demographics of "your account holders" don't match the demographics of "all business owners"), but at least you won't get increased response rates from people with extreme values.
This, incidentally, is why major consumer Internet properties like Google and Facebook are so valuable. They have data on a large swath of humanity, selected because they own a service that virtually everyone finds helpful, and so they can run experiments within that population to draw useful conclusions about human nature.
If you can't get random sampling (which is often hard when you're looking at something personally sensitive like income or net worth), the best you can do is to select your population through some means presumably uncorrelated with the quantities you're trying to measure, and then randomly sample within that. For example, if you were a bank or other financial institution, you'd have data on everyone's net worth, and you could randomly sample within the population of business owners there to draw useful statistics. There's still some sample bias (eg. perhaps the demographics of "your account holders" don't match the demographics of "all business owners"), but at least you won't get increased response rates from people with extreme values.
This, incidentally, is why major consumer Internet properties like Google and Facebook are so valuable. They have data on a large swath of humanity, selected because they own a service that virtually everyone finds helpful, and so they can run experiments within that population to draw useful conclusions about human nature.