News for hackers. Stuff that matters.
Re:A Quick Puzzle to Test Your Problem Solving by ddlatham on Thursday July 02, 2015 @09:48AM (#9818806)
I'm curious to see more about the distribution of questions and answers people had, and how the HN population may differ from the NYT's. There will certainly be self selection bias here, but if you're willing to share how you did with others, please enter it here: https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCV...

The result summary is visible here: https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCV...

The raw answers are visible here: https://docs.google.com/spreadsheets/d/1ZxR2_eOUtNLXwgKfLO1J...

Re:A Quick Puzzle to Test Your Problem Solving by veli_joza on Thursday July 02, 2015 @10:04AM (#9818912)
People familiar with unit testing and test driven development will feel at home with this kind of puzzle. That doesn't mean that they will be less biased in social/political decisions, it just means that this test will fail to prove a point.
Re:A Quick Puzzle to Test Your Problem Solving by ttctciyf on Thursday July 02, 2015 @10:13AM (#9818963)
Hah :)

Seriously, though, it seems a bit of leap from the existence of confirmation bias to explaining away the public outpourings of US Politicians about their financial crises and foreign policy disasters - in the absence of better data as to just why the given statements were made ascribing this to confirmation bias seems itself open to accusations of confirmation bias! :)

Re:A Quick Puzzle to Test Your Problem Solving by tikhonj on Thursday July 02, 2015 @02:09PM (#9820570)
I mean, it also illustrates how training and systematic reasoning can improve these things. Whether explicitly or implicitly, I picked up certain skills and procedures for problem solving (from programming and math contests) that I now use by default. Trying a bunch of examples, coming up with a hypothesis, trying to disprove it, testing edge cases…

This doesn't mean I always use these—at the very least, I have to explicitly jump into "problem solving" mode—but it means they can be useful.

It's still a meaningful difference, and could very well apply to lots of things beyond this kind of puzzle.

Re:A Quick Puzzle to Test Your Problem Solving by diminishedprime on Thursday July 02, 2015 @11:04AM (#9819346)
That's exactly what I was thinking. I (sometimes) follow TDD, and I applied it to this problem. I made sure to include negatives, 0, positives, and include primes here or there to help avoid issues with multiplication/exponentiation. After a few of these, I felt pretty confident that the rule was simple.
Re:A Quick Puzzle to Test Your Problem Solving by joezydeco on Thursday July 02, 2015 @01:25PM (#9820279)
Did you try floating point numbers? I didn't see anything in the text that said integers only.
Re:A Quick Puzzle to Test Your Problem Solving by tptacek on Thursday July 02, 2015 @03:23PM (#9821015)
I tried floating point numbers. Also, at 28 decimal places, the test breaks; it's not arbitrary precision. So, technically, the answer isn't simply "any ascending sequence of numbers".
Re:A Quick Puzzle to Test Your Problem Solving by SiVal on Thursday July 02, 2015 @04:34PM (#9821434)
I actually avoided going that far to avoid getting bad data. I was trying to answer the question, "What does the experimenter THINK his rule is?" rather than what will the computer do. Since the computer can't be infinite, it will inevitably fail with overflow, underflow, and such.

I was relieved, in fact, when it worked with negatives and floats in a "safe" range.

I also tested with 1,1,2 and 1,2,2 to make sure that the required increase applied to ALL of the values, not just a specific pair.

Re:A Quick Puzzle to Test Your Problem Solving by mjevans on Thursday July 02, 2015 @05:56PM (#9821977)
I too tried to test if it was only one pair that was significant. However I grew impatient and didn't try to come up with more tests when I thought I had a sufficient answer to explain my most vexing observation (negative, positive, positive out of combinations involving negative numbers).

The observation to brainstorm for ways of proving that a statement is in fact wrong, and exhausting them, is such an eloquent way of wording the hunt for a negative.

Re:A Quick Puzzle to Test Your Problem Solving by facetube on Thursday July 02, 2015 @09:01PM (#9822770)
The slightly-shorter 0.60000000000000000, 0.60000000000000001, 0.60000000000000002 will break it too, for what it's worth. If you punch those in to a Javascript console, you can see that the FP representation of all three is 0.6.
Re:A Quick Puzzle to Test Your Problem Solving by _asummers on Sunday July 05, 2015 @04:14PM (#9834749)
0.20000000000000000, 0.20000000000000001, 0.20000000000000002 also work.
Re:A Quick Puzzle to Test Your Problem Solving by joezydeco on Thursday July 02, 2015 @05:26PM (#9821767)
Hah! Nice catch.
Re:A Quick Puzzle to Test Your Problem Solving by tptacek on Thursday July 02, 2015 @05:34PM (#9821826)
Pentester. :)
Re:A Quick Puzzle to Test Your Problem Solving by jrapdx3 on Thursday July 02, 2015 @03:50PM (#9821181)
Yes I did, though not to 28 places as tptacek had done. With 2 to 4 place floats, including negatives, the test worked as expected.

Like others here have said it wasn't a particularly hard "rule" to figure out. Easy to immediately rule out geometric relationship as in 2^y, which didn't leave a whole lot of possibilities to test. For the commenters here, I'd attribute ease of finding the solution to familiarity with the kinds of problems that programming presents.

Which leads to the idea there's value in learning even the rudiments of programming. Logically, it should encourage better problem-solving skills in general. We might think there this has important implications for our educational systems. But I know, that's probably not realistic at all.

Re:A Quick Puzzle to Test Your Problem Solving by johnsonjo on Thursday July 02, 2015 @11:35PM (#9823286)
I know the question isn't directed at me, but I thought you might want to know. I couldn't try negatives or floating point on the iPhone. The keypad didn't have the option.
Re:A Quick Puzzle to Test Your Problem Solving by ubernostrum on Thursday July 02, 2015 @12:06PM (#9819782)
My process was:

    [3 5 7]
    [7 5 3]
    [8 4 2]
    [5 7 3]
    [1 2 3]
    [1 1 1]
    [0 1 2]
At that point I could've done some more to be really certain, but felt confident enough and guessed (correctly).
Re:A Quick Puzzle to Test Your Problem Solving by perlgeek on Thursday July 02, 2015 @02:59PM (#9820867)
So you limited yourself to non-negative integers, and had no idea what negative numbers or fractions would do.
Re:A Quick Puzzle to Test Your Problem Solving by bbcbasic on Thursday July 02, 2015 @06:19PM (#9822095)
Also he didn't know what numbers whose digits add up to 88 would do.
Re:A Quick Puzzle to Test Your Problem Solving by __david__ on Thursday July 02, 2015 @06:44PM (#9822237)
Their input on the phone wouldn't allow minus or period characters so I assumed positive integers were the set.
Re:A Quick Puzzle to Test Your Problem Solving by eli on Thursday July 02, 2015 @05:37PM (#9821846)
Based on the context of the question and the UI of the testing interface, fractions seem unlikely to be an intended part of the question. I likewise wouldn't bother testing unicode U+216x roman numerals.
Re:A Quick Puzzle to Test Your Problem Solving by Ntrails on Friday July 03, 2015 @04:51AM (#9824365)
It refused to accept both fractions and imaginary numbers. I did test negatives and zeros since that was really the only remaining set I could think of.

Most importantly I used about 6 tests (3 right 3 wrong) to come up with the answer and then did another 17 looking for the trick. After all, it couldn't just be that simple right?

Re:A Quick Puzzle to Test Your Problem Solving by ubernostrum on Sunday July 05, 2015 @12:45AM (#9832497)
The presentation of the problem -- and I know that trusting the problem state is unwise sometimes in cognitive-bias tests, since many such tests are actually designed to be "we said we were asking X but actually meant Y" -- indicated a simple rule, rather than one which would behave differently on different classes of numbers.

So after the tests listed above I felt confident enough to guess.

Re:A Quick Puzzle to Test Your Problem Solving by jonkiddy on Thursday July 02, 2015 @01:32PM (#9820330)
[16 32 64] [0 0 0] [1 2 3] [3 2 1] [-1 0 1] [3 5 7] [-2 -1 0] [111 54231 9999999999999999999999]

At that point, I correctly answered the question.

Re:A Quick Puzzle to Test Your Problem Solving by jws on Thursday July 02, 2015 @03:02PM (#9820894)
Similar to mine, but we both actually failed. Real numbers are accepted and we did not test that.
Re:A Quick Puzzle to Test Your Problem Solving by WildUtah on Thursday July 02, 2015 @06:54PM (#9822285)
It accepts floating point numbers. To approximately double precision.

But it accepts zero percent of real numbers.

Re:A Quick Puzzle to Test Your Problem Solving by eru on Friday July 03, 2015 @12:38AM (#9823500)
I can nitpick better than that!

To talk about a certain fraction of real numbers you have to have a distribution over them. In general we take the uniform distribution if no distribution is explicitly given. That doesn't work for real numbers (it doesn't even work for natural numbers). (See https://math.stackexchange.com/questions/14777/why-isnt-ther...)

If there's no implicit default distribution, we have to pick on. I can pick one where they cover an arbitrary high percentage of real numbers..

Re:A Quick Puzzle to Test Your Problem Solving by elnion on Friday July 03, 2015 @02:55PM (#9827306)
Down the rabbit hole of pedantry: we don't need a distribution, just a measure, if we want to talk about how many reals it accepts. The Lebesgue measure is implied on the Reals if none is given, and the computable reals have measure zero.

We can't reasonably talk about a percent coverage, since the Lebesgue measure of the reals is infinite, but as a non-technical description, 'zero percent' is morally equivalent to saying it only covers a measure-zero set.

Re:A Quick Puzzle to Test Your Problem Solving by profinger on Thursday July 02, 2015 @03:26PM (#9821035)
I did this plus also testing [1 1.5 2]
Re:A Quick Puzzle to Test Your Problem Solving by rahimnathwani on Thursday July 02, 2015 @02:10PM (#9820578)
I was a little less systematic, but still had a positive:negative ratio of 1:2 before I submitted my answer.

4,6,8 Y 1,1,1 N 1,2,3 Y 1,6666,8777 Y 1,0,1 N 3,2,1 N 3,2,3 N 5,6,4 N 7,5,6 N

Re:A Quick Puzzle to Test Your Problem Solving by aptwebapps on Friday July 03, 2015 @12:04AM (#9823381)
Exactly. I'm just as 'No' averse as the next person but the 'No' I'm averse to is the one where you make wrong assumptions and it comes back to haunt you afterwards.
Re:A Quick Puzzle to Test Your Problem Solving by benihana on Thursday July 02, 2015 @12:59PM (#9820126)
I was just writing this out! I believe practice in unit testing is what let me get this question right. I almost submitted n^1, n^2, n^3 before I realized I didn't get any wrong answers and something didn't feel right. I credit this 'instinct' to having written thousands of tests, and trying to make them fail to make sure my tests weren't lying to me.
Re:A Quick Puzzle to Test Your Problem Solving by Natsu on Thursday July 02, 2015 @08:40PM (#9822683)
It's funny. Even though I got the right answer, looking back, I still see many holes in my tests. I only tested positive and negative integers. I didn't even think to try fractions, decimals, hex values, text, etc. and I was already expecting a confirmation bias test.
Re:A Quick Puzzle to Test Your Problem Solving by LoSboccacc on Thursday July 02, 2015 @01:58PM (#9820504)
got tricked even if I tested 1, 1, 1
Re:A Quick Puzzle to Test Your Problem Solving by tjohns on Thursday July 02, 2015 @05:13PM (#9821676)
1, 1, 1 was actually the test case that clued me in that n^1, n^2, n^3 couldn't be the right solution.
Re:A Quick Puzzle to Test Your Problem Solving by tikhonj on Thursday July 02, 2015 @09:31PM (#9822885)
n=1?

But I totally see why it would cause you to pause and rethink your original idea :).

Re:A Quick Puzzle to Test Your Problem Solving by techonup on Thursday July 02, 2015 @10:21PM (#9823032)
The point is that it didn't work, so exponentiation was out the window.
Re:A Quick Puzzle to Test Your Problem Solving by tikhonj on Friday July 03, 2015 @01:08AM (#9823621)
My point was that 1,1,1 is 1^1,1^2,1^3 and actually fits the pattern :).
Re:A Quick Puzzle to Test Your Problem Solving by firethief on Friday July 03, 2015 @01:57AM (#9823774)
Right - by fitting the pattern but failing the actual criterion, it disproves the consecutive exponents hypothesis.
Re:A Quick Puzzle to Test Your Problem Solving by jerf on Thursday July 02, 2015 @12:34PM (#9819979)
This is now the fourth time I've seen this exact rule. Another example: https://www.youtube.com/watch?v=vKA4w2O61Xo

It's probably not "fair" to say I got it in zero... but I did. :)

Now that the NYT has done it, this puzzle has probably attained enough popularity now that you really ought to change it up a bit now if you're going to run it yourself. Granted, the space of hypotheses as simple as "increasing/decreasing" is pretty small, but your ability to fool people with the first sample run is almost unbounded, so that helps.

Re:A Quick Puzzle to Test Your Problem Solving by Verdex on Thursday July 02, 2015 @03:41PM (#9821122)
Similar story here. I got it in zero because this problem shows up at the early part of HPMOR.

I suspect that the basic idea behind it is about right (people who insist on failures before committing to a theory will probably do "better"). But it seems to me that this test will be best at selecting people who've seen it before and can pretend they didn't (or even remember to ask negative questions when someone asks you to guess three numbers to get the job).

Re:A Quick Puzzle to Test Your Problem Solving by MichaelDickens on Thursday July 02, 2015 @07:24PM (#9822410)
Yeah, I'd like to see other sorts of questions that you can only get right by looking for disconfirming evidence, but that don't have anything to do with choosing a sequence of numbers.
Re:A Quick Puzzle to Test Your Problem Solving by larkspur on Thursday July 02, 2015 @08:35PM (#9822664)
If you look at the raw answers, a lot of people who guessed the wrong answer according to: "What answer did you give?" said they got it right:

Right? What answer did you give?

Yes, double the previous number

Yes, Each number much larger than the previous

Yes, sequence must always increase by 1

Yes, Powers of 2

Yes, h = 2n, i = 2(n+1), j = 2(n+2) . n is an integer

Yes, Powers of 2

Re:A Quick Puzzle to Test Your Problem Solving by abandonliberty on Thursday July 02, 2015 @03:29PM (#9821050)
There's a selection bias - those of us who got it right are more likely to fill that out (:

As a result we can't really rely on overall accuracy, but we can break it out by yes/no to account for the selection bias to get a profile for how a HN correct and incorrect differ.

Re:A Quick Puzzle to Test Your Problem Solving by nzealand on Thursday July 02, 2015 @12:20PM (#9819890)
This crowd needs one more question:

How many "I know the answer, but can I find a flaw in their code" questions did you ask?

Re:A Quick Puzzle to Test Your Problem Solving by stillsut on Thursday July 02, 2015 @12:38PM (#9820001)
to be pedantic: "monotonically increasing" is incorrect. It should be "strictly increasing"
Re:A Quick Puzzle to Test Your Problem Solving by fenomas on Thursday July 02, 2015 @10:02AM (#9818903)
Cool idea, thanks!

Personally, I spent most of my answers playing with the inputs. The form was happy to report that "1e1, 15, 0x10" was a valid sequence. :D

Re:A Quick Puzzle to Test Your Problem Solving by elevensies on Thursday July 02, 2015 @11:28AM (#9819522)
This is a good idea. I submitted the survey but here are my tests: http://imgur.com/hYKudyl

FWIW, I've seen this kind of game before and I was expecting it to be something simple.

Re:A Quick Puzzle to Test Your Problem Solving by Jugurtha on Thursday July 02, 2015 @02:07PM (#9820552)
My answer was: "The sequence is of an increasing real variable, where each subsequent value is greater than the preceding value. It's monotonically varying."

When I clicked "I think I know it", nothing happened. I don't want to click their "I don't want to play; just tell me the answer". But it seems like the right answer. I can't answer your form question if it is the right anwer, since I haven't clicked on their link and don't know for a fact whether it is or not.

Although I used the wrong term, it's strictly increasing.

Re:A Quick Puzzle to Test Your Problem Solving by wzdd on Thursday July 02, 2015 @11:39AM (#9819593)
Your last question is "have you seen a test about confirmation bias before?" But the text of this test says that it's about "why no-one likes to be wrong", which means it's pretty obviously about confirmation bias (and therefore that the test-taker should be wary about just confirming their first intuition).
Re:A Quick Puzzle to Test Your Problem Solving by scuba_man_spiff on Thursday July 02, 2015 @11:32PM (#9823278)
[ 456 , 42 , 48 , No ] [ 1 , 2 , 3 , Yes ] [ 3 , 2 , 1 , No ] [ 45 , 46 , 47 , Yes ] [ -1 , 999 ,1238978 , Yes ] [ 1+1 , 1+2 , 1+3 , Fail ] [ 7 , 654 , 653 , No ] [ 7 , 654 , 655 , Yes ] [ 1.1 , 1.2 , 1.3 , Yes ] [ 1 , e , pi , Fail ] [ 1 , e , 42 , Fail ] [ -1.1 , -1.0 , -0.9 , Yes ] [ 2i , 3i , 4i , Fail ] [ 1 , e , pi , Fail ] [ 0 , 1 , 2 , Yes ] [ -1 , 0 , 1 , Yes ]

If anyone's tallying

Re:A Quick Puzzle to Test Your Problem Solving by pitchups on Thursday July 02, 2015 @11:45AM (#9819641)
Intersting that the split of correct/wrong answers from the HN crowd is 78%/22% - the exact opposite of the general population : 22% / 78%! The HN community does think different :)
Re:A Quick Puzzle to Test Your Problem Solving by teahat on Thursday July 02, 2015 @12:46PM (#9820050)
The article doesn't actually specify that 78% guessed correctly or incorrectly, just that 78% guessed without ever entering an invalid sequence.
Re:A Quick Puzzle to Test Your Problem Solving by philh on Thursday July 02, 2015 @12:45PM (#9820040)
If you're taking that from the article, I think you misread it.

> Remarkably, 78 percent of people who have played this game so far have guessed the answer without first hearing a single no.

Some of those 78% probably got it right, and some of the remainder would have got it wrong.

Re:A Quick Puzzle to Test Your Problem Solving by jessaustin on Thursday July 02, 2015 @07:26PM (#9822417)
Some of those 78% probably got it right...

In what sense? How did they know that e.g. (1, 1, 1) didn't work, unless they tried it?

Re:A Quick Puzzle to Test Your Problem Solving by philh on Thursday July 02, 2015 @07:45PM (#9822488)
In the sense that they guessed the rule correctly. The correct answer was the dominant hypothesis in my mind before I tried any sequences, and others here report the same.
Re:A Quick Puzzle to Test Your Problem Solving by snowwrestler on Thursday July 02, 2015 @12:39PM (#9820009)
Or at least, the portion of the HN community who reported their results, reported that they think different. :-)
Re:A Quick Puzzle to Test Your Problem Solving by JFoss117 on Thursday July 02, 2015 @03:00PM (#9820884)
Yep, I'd guess this is a massive bias. The people who get it right are much more apt to report and show how smart they are
Re:A Quick Puzzle to Test Your Problem Solving by Yizahi on Friday July 03, 2015 @07:53AM (#9824905)
It just means that this test was described at least several times in different "computer" media over last few years and almost everyone has read about it. Same thing with all "logic" puzzles.
Re:A Quick Puzzle to Test Your Problem Solving by maxerickson on Thursday July 02, 2015 @11:12AM (#9819414)
I wanted to answer "Probably" to the last question about having seen a similar question before.

I don't specifically recall seeing one, but it is likely that I have.

Re:A Quick Puzzle to Test Your Problem Solving by hammock on Thursday July 02, 2015 @12:54PM (#9820100)
Guessed after just one "No"

  3  9 27  yes  (is it exponential series?)
  4 16 64  yes  (is it only odd numbers?)
  5  7  9  yes  (is it any numbers of the same parity?)
  6  7  8  yes  (is it any set of increasing numbers?)
  6  7  6  no   (just to confirm that it's x<y<z, and not something like x<=y<=z)
Re:A Quick Puzzle to Test Your Problem Solving by Retric on Thursday July 02, 2015 @01:14PM (#9820222)
6,7,7 or 1,1,1 satifies x<=y<=z not 6, 7, 6
Re:A Quick Puzzle to Test Your Problem Solving by Falcon9 on Thursday July 02, 2015 @01:00PM (#9820138)
-2 -4 -8 (is it increasing distance from zero?)
Re:A Quick Puzzle to Test Your Problem Solving by hammock on Thursday July 02, 2015 @01:04PM (#9820157)
You're right, I made the assumption that our domain was natural numbers- which seems to be an uncommon one so far among the HN crowd
Re:A Quick Puzzle to Test Your Problem Solving by Myrmornis on Thursday July 02, 2015 @09:46PM (#9822933)
It would be relevant to include a histogram of #yes_answers - #no_answers in the summary, to test whether people are biased towards positive rather than negative tests. I think the raw data suggests that it does although I totally failed to create a histogram in google spreadsheets within 5 minutes.
Re:A Quick Puzzle to Test Your Problem Solving by beatbrokedown on Thursday July 02, 2015 @11:01PM (#9823174)
My fuzzing went:

8 4 2 (no) 1 2 3 (yes) 1 1 1 (no) 1 100 123 (yes) 1.0 1.1 1.2 (yes)

answer: incrementing numbers

Re:A Quick Puzzle to Test Your Problem Solving by eru on Friday July 03, 2015 @12:32AM (#9823479)
Thanks for setting up the survey!

Could you perhaps move to a bar chart instead of pie charts?

Re:A Quick Puzzle to Test Your Problem Solving by ddlatham on Friday July 03, 2015 @12:50AM (#9823552)
Don't think that's an option for Google docs - but feel free to take the data and share one.