http://neilbowers.wordpress.com/2008/08/11/pink-box-testing/
I had a lecturer (a Brit) who would insert the phrase 'Rule Britannia' into lecture notes to see if anyone ever read them.
There is also an urban myth about some entrance exam for Police/Army/Pilots etc where the instructions say read all the questions first. And the last question says, do not answer any questions just 'do some arbitrary thing' - to check wether you follow instructions properly.
Personally, I don't think there is a lesson to be learned here. It amounts to nothing more than a lame practical joke.
The one we had was a set of increasingly complex instructions, and the last instruction saying to just sit there quietly. So you would be working hard at solving the problems, and various people would just be sitting there with a smile on there face. You knew these people, since they were your classmates, so they weren't savants, so how did they get through these problems so quickly. Eventually each one of you realise, and look for the trick, getting to the last question. There is always someone who just keeps going.
2) Even if it did teach you to follow instructions, is that a good lesson? I think I've only gotten to where I am in life by explicitly NOT following instructions.
#3 Just happened to be the toughest material of the class, where a mastery of both Haitian witchcraft and the Coriolis effect were needed.
I have actually experienced this kind of test. in my case, it was coupled with telling the test participants that they only have a short amount of time to complete the test, thus making it even more likely that the test participants would start answering questions immediately rather than following instructions to the letter.
Particularly when this extra time limit trick may be used, I'm not sure what value the test really had. Supposedly, even the police and armed forces are looking for people with initiative who can think creatively and not just mindless automatons who follow orders without question. This kind of test would not seem to align with that goal.
Perhaps creativity vs. "automaton-hood" is not the dichotomy being tested and the value of the test really is more to do with following instructions when not doing so could kill you. But the sceptic/rebel in me wonders about those situations where following the instructions may well get you killed. I am not in the military or law enforcement and for better or worse, past experiences have taught me to be suspicious of both.
When Google asks prospective PMs what a binary search or hashtable is, it's not because they actually expect product managers to ever write one. It's because those are a proxy for general CS knowledge; it shows that either they've come out of an academic CS department or they hang around engineers enough to know the basic terminology. This in turn is a proxy for "Won't demand unreasonable features from engineering and can communicate reasonably well with them." This is rather important for a tech company, as anyone who's ever worked under someone with no clue about technology could tell you.
Similarly, when companies ask how many gas stations there are in the U.S, they don't actually care what the answer is. Rather, they want to see how you respond when put in a situation where you obviously don't have enough information to come up with a precise answer. They want to see what sort of background information you draw on to make an estimate; this is often a proxy for how open and aware you are to new experiences. They want to see you combine whatever background information you have to solve a problem. They want to see you self-check and see if your answer seems reasonable.
Yet for some reason, it's clever when Van Halen does it and pedantic when it happens in a job interview...
The "how many gas stations in the US" question is nothing like the brown M&Ms test. It is not testing anything nearly so specific, nor is it as easily confirmed by experience. At best you're taking a subjective interpretation of a creative answer and comparing it to subjective performance down the road. And even if you don't consider the inherent subjectivity, it's hard to document a reasonable number of positive and negative test cases; and even then controlling for other variables is essentially impossible.
Van Halen used the test to predict performance conditions for a specific date and venue. The gas stations answer is used to predict the behavior of a person for a significant period of time.
Enlightenment! There is a method to the madness of those insane autocrats who demands ridiculous things...
The thing is, it is madness all the same...