Re:Results for: Pick a number from 1 to 10
by alecperkins on Monday March 28, 2011 @12:39AM
(#2377173)
I just updated the results page with a breakdown per-interface. There are more metrics involved, but it'll be a little while before those get pulled together. (The volume of data was unexpected, to say the least, so there weren't any stats helpers in place prior to "launch".)
As for "revealing" the answers ahead of time, the whole question of "Pick a number from 1 to 10" and the common result being 7 is well known (ish), so I'm not too worried. Plus, the results aren't visible until you choose. PLUS, there were over 80k votes before that first graph was available, and over 140k by the time I put the per-interface breakdown up.
Hopefully, the volume will outweigh the people who try to manipulate it. (There's also IP tracking, which will eventually be used to help filter out anomalies, in addition to geolocating.)
I know, it's not exactly precise science, but it's a fun experiment that was simple and kept the friction to a minimum. I'm not exactly trying to write a paper. :)
Thanks to everyone who voted and spread the word, especially jedberg who posted it to Reddit.
Re:Results for: Pick a number from 1 to 10
by albertzeyer on Monday March 28, 2011 @05:31AM
(#2377732)
43 is actually not bad. I would think that most people would pick something somewhere in the middle. A somewhat similar distribution as with the 1-10 results. Odd numbers more often than even numbers. Prime numbers even more often.
Re:Results for: Pick a number from 1 to 10
by travisglines on Sunday March 27, 2011 @11:50PM
(#2377061)
It seems like publicizing some preliminary results of the study before the study is complete would put the legitimacy of the final results very much so at risk.
Re:Results for: Pick a number from 1 to 10
by travisglines on Monday March 28, 2011 @01:28AM
(#2377260)
If (as I'm sure you do) you have the times along with the votes it would be cool to see if there was actually a bias when comparing the before response and after response datasets.
Re:Results for: Pick a number from 1 to 10
by alecperkins on Monday March 28, 2011 @12:42AM
(#2377180)
That bit of "flair" was added almost on a whim. I'm glad I did add it. I think that helped the experiment get the attention it did, especially when it got onto Reddit. It's funny how a simple factor can (probably) have such a big impact.
Also, the unintentional "mystery" of not including any information about the who or why on the site itself may have helped. I noticed many comments on HN and Reddit that speculated about each, which no doubt increased the activity profile. A little bit of personality goes a long way.
Re:Results for: Pick a number from 1 to 10
by bradly on Monday March 28, 2011 @12:00AM
(#2377085)
Was this only posted to HN? If it was posted on other sites
it would neat to check the numbers against refers to see if HN is better at picking random numbers than, say, Reddit.
Re:Results for: Pick a number from 1 to 10
by alecperkins on Monday March 28, 2011 @12:21AM
(#2377136)
It ended up getting posted to Reddit, and has been making the rounds elsewhere. (Over 100,000 uniques, holy cow!)
Unfortunately, the referrer wasn't tracked. I wasn't expecting to get enough hits for it to matter. The expectation was for maybe 100 over a week. IP addresses are tracked, so I'll eventually do a location breakdown.
Re:Results for: Pick a number from 1 to 10
by djahng on Monday March 28, 2011 @12:35AM
(#2377166)
The site asks you to pick a random number, not mentioning anything about trying to achieve a specific distribution.
EDIT: Downvotes and no comments, nice. You guys seem to be misunderstanding something. If asked to picked any number ("random") between 1 and 10, to suggest that the outcome is "wrong" because the distribution isn't uniform doesn't make any sense at all. It would be a completely different experiment if we were asked to pick a number between 1 and 10 such that the outcome after x number of independent trials has a uniform (or gaussian/exponential/etc) distribution. This seems to be what some people are assuming.
Re:Results for: Pick a number from 1 to 10
by djahng on Monday March 28, 2011 @01:32AM
(#2377266)
The point of probability is to describe uncertain events. Coin flips aren't described by uniform distributions, they're binomial. Human intelligence can be modeled by a Gaussian distribution. And this site's experiment seems to suggest that picking a number between 1 and 10 can be modeled by a bimodal random variable with means around 4 and 7. Point being: random doesn't necessarily mean uniform.
Re:Results for: Pick a number from 1 to 10
by sesqu on Monday March 28, 2011 @04:36AM
(#2377652)
No, but random does mean uncertain, and the uniform distribution has maximal entropy. In other words, uniformly distributed numbers are most random, colloquially, and so you can improve your ability to generate random numbers by sampling from a more unform distribution.
Also, just because intelligence is described with normal distributions, it is not at clear that it can be.
Re:Results for: Pick a number from 1 to 10
by sp332 on Monday March 28, 2011 @01:32PM
(#2379208)
This comes from the fact that the first run of six 9's in the digits of pi comes much sooner than expected: http://www.geom.uiuc.edu/~huberty/math5337/groupe/digits.htm... search for 999999 - you'll see it's only 10 lines down. When discovered, this caused a lot of people to conclude that the digits were not random. Of course, with 5 trillion digits now calculated, there is still no bias found toward any pattern. So it is very hard to tell!
Re:Results for: Pick a number from 1 to 10
by Raphael on Monday March 28, 2011 @01:24AM
(#2377253)
The larger the sample, the more likely a truly random input approaches a uniform distribution. Yes, a coin flip could come up heads 10000 times in a row, but it just doesn't happen.
Re:Results for: Pick a number from 1 to 10
by vladd on Monday March 28, 2011 @04:01AM
(#2377576)
Benford's law applies to those cases where you can assume that the logarithms of the numbers are uniformly distributed.
While it's perfectly reasonable to assume that for, let's say, Microsoft's yearly revenue, Apple's stock price, or a number series with x% average growth (for example, inflation-affected prices), it has nothing to do with this experiment.
Re:Results for: Pick a number from 1 to 10
by kahawe on Monday March 28, 2011 @05:36AM
(#2377750)
Strangely enough, I got asked to pick a number between 1 and 10 a few days ago in an Irish pub in Germany. Looks like this was a LOT more popular than expected!
For the record, I picked 9 - much to the dismay of the person asking me.
Re:Results for: Pick a number from 1 to 10
by spravin on Sunday March 27, 2011 @11:52PM
(#2377069)
The positioning of the dropdown clearly has a lot of influence on the results. People are horrible random number generators! In this case, people are unlikely to choose the ends (1/9) or the center (5) leading to a bimodal distribution with two modes at 4 and 7.
Re:Results for: Pick a number from 1 to 10
by kmiyer on Monday March 28, 2011 @12:33AM
(#2377161)
Note however that different types of input systems were used. Refreshes show the following different types:
* A slider that need to be dragged to choose a value.
* A list of numbers shown with highlighting on hover.
* An input cursor that on hover changes to a list of numbers (within a curved border this time).
* A text field (enclosed in a circular border), allowing manual input.
I'm quite interested in how results varied depending upon input type (possibly more interesting than seeing how they vary by referrer).
Re:Results for: Pick a number from 1 to 10
by klochner on Monday March 28, 2011 @02:36AM
(#2377403)
I picked 3 because it's in the easiest number for me to strike (text box version).
My mouse is typically in the middle of the screen, closest to the higher-scoring numbers in the graphical version.
I'd like to see the results from the same test where you have to type the number as a word for the textbox (e.g., three), and where the numbers aren't laid out in the same numerical order for each user in the graphical version.
Re:Results for: Pick a number from 1 to 10
by vidyesh on Monday March 28, 2011 @12:34AM
(#2377163)
Seems like people are so number centric and go for the universal lucky number. Not sure why.
How did this help? Were you able to do any behavior analysis based on this small experiment?
I would suggest, go on a weekly result and keep tabs on result variation.
Try posting it on various/diverse communities. This might help to understand alot about visitor behavior on various websites as you can track referring websites too.
If possible graph the picks depending upon the referring websites. Or even a graph based on region, tracking via IP should be easy.
Re:Results for: Pick a number from 1 to 10
by orijing on Monday March 28, 2011 @12:11AM
(#2377110)
The problem is that it didn't ask us to choose the numbers uniformly. Perhaps I picked from a Gaussian distribution with mean 5 and a strict cut-off at 1 and 10.
I don't think anything can be concluded unless it asked for "choose uniformly at random among the 10 numbers," if we want results robust against question biasing.
Re:Results for: Pick a number from 1 to 10
by Eliezer on Monday March 28, 2011 @01:43AM
(#2377286)
"Choose randomly" is generally understood to mean "choose randomly from a probability distribution containing as much entropy as possible given the problem." I mean, I could choose from the probability distribution {0.001, 0.001, 0.998} but it's commonly understood that this isn't what's meant by the word "random".
Re:Results for: Pick a number from 1 to 10
by philh on Monday March 28, 2011 @12:39PM
(#2378972)
The results for 'select' seem strange. That's the only place where random specified/not makes much difference. It also indicates that a lot more people picked numbers under 'select/specified' than any other condition.
Have you checked your logs for possible vote stuffing?
Re:Results for: Pick a number from 1 to 10
by adleberg on Monday March 28, 2011 @12:21AM
(#2377132)
Is it wise to publish updated results while the data is still being gathered? Random selection should be blind and independent and its quite possible people may change their vote to favor an 'unpopular' number or vice versa.
Re:Results for: Pick a number from 1 to 10
by kipwork on Sunday March 27, 2011 @11:39PM
(#2377036)
I think that the reason that 7 is so high is because it is commonly believed that 3 will is the first one you think of. As a result people choose to pick something higher, hence 4 and 7 being the top two.
Re:Results for: Pick a number from 1 to 10
by metachris on Sunday March 27, 2011 @11:50PM
(#2377063)
I was talking with a friend when I visited the website, and just completely unconsciously entered any number. It was 7. After submitting I wondered why I did that without even thinking about it.
Re:Results for: Pick a number from 1 to 10
by chromejs10 on Tuesday March 29, 2011 @01:29AM
(#2381484)
I'm surprised the number 1 wasn't higher up because of the slider (since it defaults to 1). I guess people aren't as lazy as I thought they were and actually took the effort to move the slider :P
Re:Results for: Pick a number from 1 to 10
by davidk0101 on Monday March 28, 2011 @04:38AM
(#2377655)
I don't get it. How is the selector relevant to what people choose? You might as well have put a different colored rabbit somewhere on the page and measured responses that way.
Re:Results for: Pick a number from 1 to 10
by alecperkins on Monday March 28, 2011 @02:22PM
(#2379394)
Momentary stupidity on my part. I told it to cache the expire time instead of the actual value. Oops. Go figure, it was the one time I think "Eh, this is a small change. I don't need to go to a testing version first."
As for "revealing" the answers ahead of time, the whole question of "Pick a number from 1 to 10" and the common result being 7 is well known (ish), so I'm not too worried. Plus, the results aren't visible until you choose. PLUS, there were over 80k votes before that first graph was available, and over 140k by the time I put the per-interface breakdown up.
Hopefully, the volume will outweigh the people who try to manipulate it. (There's also IP tracking, which will eventually be used to help filter out anomalies, in addition to geolocating.)
I know, it's not exactly precise science, but it's a fun experiment that was simple and kept the friction to a minimum. I'm not exactly trying to write a paper. :)
Thanks to everyone who voted and spread the word, especially jedberg who posted it to Reddit.
constructive criticism ... for science
Also, the unintentional "mystery" of not including any information about the who or why on the site itself may have helped. I noticed many comments on HN and Reddit that speculated about each, which no doubt increased the activity profile. A little bit of personality goes a long way.
Unfortunately, the referrer wasn't tracked. I wasn't expecting to get enough hits for it to matter. The expectation was for maybe 100 over a week. IP addresses are tracked, so I'll eventually do a location breakdown.
EDIT: Downvotes and no comments, nice. You guys seem to be misunderstanding something. If asked to picked any number ("random") between 1 and 10, to suggest that the outcome is "wrong" because the distribution isn't uniform doesn't make any sense at all. It would be a completely different experiment if we were asked to pick a number between 1 and 10 such that the outcome after x number of independent trials has a uniform (or gaussian/exponential/etc) distribution. This seems to be what some people are assuming.
Also, just because intelligence is described with normal distributions, it is not at clear that it can be.
While it's perfectly reasonable to assume that for, let's say, Microsoft's yearly revenue, Apple's stock price, or a number series with x% average growth (for example, inflation-affected prices), it has nothing to do with this experiment.
For the record, I picked 9 - much to the dismay of the person asking me.
I just clicked whatever was closest because I wasn't even thinking about the exercise.
My mouse is typically in the middle of the screen, closest to the higher-scoring numbers in the graphical version.
I'd like to see the results from the same test where you have to type the number as a word for the textbox (e.g., three), and where the numbers aren't laid out in the same numerical order for each user in the graphical version.
I would suggest, go on a weekly result and keep tabs on result variation.
Try posting it on various/diverse communities. This might help to understand alot about visitor behavior on various websites as you can track referring websites too.
If possible graph the picks depending upon the referring websites. Or even a graph based on region, tracking via IP should be easy.
For those who missed the Parent post, check here http://news.ycombinator.com/item?id=2375149
I won! Yay.
I don't think anything can be concluded unless it asked for "choose uniformly at random among the 10 numbers," if we want results robust against question biasing.
PS: I chose 2. Eat that, seveners!
10 was a good guess.
Have you checked your logs for possible vote stuffing?
Sort of looks to me like people avoid the extremes and also 5 (possibly because it's "right in the middle" and so perceived as "less random"?).