Re:Pairwise and Y Combinator: Do you have what it takes to b...
by euccastro on Friday March 30, 2007 @07:54PM
(#7728)
TBH, my feeling about this stuff is that it belongs in nazi research labs and Cosmopolitan magazines, rather than as a screening tool in business where you want the best people. If I was considering applying to anything (Y, a job at a company), being presented with this would disappoint me and set off Dilberty alarms.
Now, this is fun stuff to burn some minutes while procrastinating! Here are my results:
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by bootload on Friday March 30, 2007 @07:42PM
(#7726)
'You may have someone with a very high IQ but has low interpersonal skill .. cannot relate to other people very well. Cannot understand other people. This can be a real liability for organisations who are looking for leadership .. looking to promote people with high intelligence' [0]
'... all we'll care about is one measure: how close one comes
to the best founders. We don't care what atoms are in that
molecule ...'
But testing for characteristics of 'high achievers' then correlating test scores against potential applicants is fraught with problems. Is the correlation against the best enough? [1]
I'm sure you might get some correlation but it's pretty unimaginative [2]. So what could be a better tool? What about direct testing of skill? Wouldn't a better technique be simulation? Why not build a 'startup simulator' where applicants are given tools to simulate tasks they actually will have to do? Then you have a controllable scenario where you really can measure results against successful founders.
You could test
- the conception of an idea
- the building of a (simple) prototype
- quickly find an audience
- find a way to make money off it
Wrapped up in a framework [3] where you test the execution of these tasks you could get a better grasp of the
- skills
- determination
- entrepreneurial audacity
- passion
- humour
- leadership skills needed to succeed.
- risk taking, emotional intelligence
There is a long history of simulators in testing & refining of skills, competency and execution are required. Even Captain Kirk at Starfleet Academy trained, passed (and cheated) on simulators.
Who knows it might even be fun.
Reference
[0] Professor Con Stough, Brain Sciences Institute, 'Director, Centre for Neuropsychology, Swinburne Institute of Technology'
[1] But I do like the simplicity of just looking at a simple set of parameters to make a Gladwellian decision.
[2] In a competitive environment the difference between getting the attention of the right candidates could be the tools used to measure how they stack up against the best .. but actually doing something, learning and getting some feedback.
[3] The framework could be a game, or simply a panel of dials. The key thing is you can have a back room with founders, past entrepreneurs twiddling the dials checking to see in RT how candidates handle things like a person leaving, stressing system etc (like the LEM tester in the Apollo missions). It also allows for candidates to learn from failure with less risk & pass on the best decision making skills to candidates that have passed.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by bootload on Saturday March 31, 2007 @10:07PM
(#7927)
'... How about doing this for real? ... Reality is the superior test of whether you are an entrepreneur ...'
True . Ideally yes. But it there may be times when you want to test skills or sub sections of skills, NOW. I've run 8 miles in 2 hours in the past but does that mean I can do it now?
I think the bit I was addressing was suggesting a technique to scale selection through automated means by testing against the best. I agree reality is a better test. But you miss the point about simulation. With simulation you can
- test/be tested/train in compressed time
- test/be tested/train with different scenarios
- test/be tested/train individual components of the process
- test/be tested/train with lower costs
- test/be tested/train in controlled circumstances to allow better measurement
'... having done something is better than simulating it ...'
Pilots fly planes but they also train and test their skills in simulation due to cost, risk and a lot of other factors. People who do things, need ways to test their skills, improve and be tested. Why not startups? In the case of YC, 'done', 'doing' may not be controlled enough to measure against the best (conjecture).
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by amichail on Friday March 30, 2007 @05:49PM
(#7690)
Is this comparison with respect to all Y combinator founders or only with respect to those who have succeeded? Have you identified any personality differences between those who succeeded and those who did not?
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by notabel on Friday March 30, 2007 @08:23PM
(#7733)
I found it relatively inaccurate, but oddly, it was always very precise; i.e. when it got things wrong, it got them profoundly, diametrically wrong.
One thing I wonder is whether there is any logic to deal with confounding variables--for instance, if someone picks every photo that has a bike in it, maybe they just like bikes, and those data points should be excluded.
On the whole, though, the pairwise people have probably as good a shot at anyone at making a not-suck personality analyzer.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by zaidf on Friday March 30, 2007 @08:28PM
(#7734)
That's how I answered most questions. I like chocolate; I like nature; I like neater pictures over complex ones. I have no idea what all that means - may be someone does.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by Tichy on Saturday March 31, 2007 @06:22AM
(#7803)
Absolutely ridiculous test. There weren't even any books in it, for example, yet they claim to be able to tell if I am into reading books or not? Sorry, but I don't believe their tests are scientifically sound. The sheer number of traits they claim to identify - almost as many traits as there were "questions", I don't think it can work.
Why don't you just use a classic IQ test if you are into that kind of stuff, or the Myer-Briggs test?
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by pg on Friday March 30, 2007 @05:37PM
(#7688)
I think they can do better if they make you look at more images, but for this online version they wanted to strike a balance between that and having people actually finish.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by domp on Friday March 30, 2007 @05:43PM
(#7689)
Yeah I can see that. I found that I had no preference in many of the image comparisons so I just clicked one.
Did they simply take the average result from the past YComb people or was there more to it? (btw sorry for posting this topic before you!)
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by volida on Friday March 30, 2007 @07:07PM
(#7718)
ok, my observation is that, the GREEN of most attributes is somewhere in the middle (with some nearer to some side), which means that there is a diversity between to those who you are compared, with 3/4 of the samples to define that side. This means that someone can be in right opposite but it doesn't mean that personality is excluded.but be in the 1/4 of the samples. So, with all this diversity, how valid and helpful is this test?
I think the attributes of the test, should be more questions of type, what you do in specific situations, and be compared to what YC would prefer as answer.
update
or am I wrong, and GREEN is the range of answers of the samples?
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by nickb on Sunday April 01, 2007 @02:55PM
(#8039)
This is in the same category as astrology. 'The Brain' got less than 50% things about me correct. It said that I like dogs better than cats, not true. It said I like to work individually, not true... I prefer working as a part of a team.
The premise that pictures and the choice of pictures determine anything about yourself is simply silly. I have different experiences than someone else. I have been to and seen different places than someone else. Different imagery will provoke different emotion in different people.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by jamiequint on Friday March 30, 2007 @07:57PM
(#7729)
This is interesting, I was on the same side in every category, and it seems fairly accurate, although I found some things remarkable far off, most notably "like to read" where I scored a decisive: "aren't much into books."
I found it weird that Y Combinator founders generally fell in the "aren't much into books" category.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by pg on Friday March 30, 2007 @06:09PM
(#7698)
To be honest, not much. We may use Pairwise seriously in the next round, but this is just a prototype.
In particular, we don't care much about the specific metrics this test tries to measure. The Pairwise guys chose those; we had nothing do to with it. When we use this for real, all we'll care about is one measure: how close one comes to the best founders. We don't care what atoms are in that molecule.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by amichail on Friday March 30, 2007 @06:22PM
(#7702)
Even if we take it as a given that a test like this can work well, I suspect that test takers will become more sophisticated at choosing images to get desired outcomes.
And so the test will need to get more sophisticated in turn by finding even more non-obvious yet discriminating pairs of images.
This is sort of like the constant battle between those creating spam filters and spammers.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by pg on Friday March 30, 2007 @06:26PM
(#7704)
The difference between this and spam is that the cycle time is too long for people to learn efficiently how to beat the system. A spammer can write an email and send it to his gmail acct and know in 30 sec if it beats the filter. We only accept applicants every 6 months.
Does anyone remember The Spark's gender test? It's a specific case of this kind of test: http://community.sparknotes.com/gender/ (use bugmenot.com for login)
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by far33d on Friday March 30, 2007 @08:03PM
(#7731)
Weird. I'm not a smoker, but I was, for a LONG time.
And it knew that. That's really strange and freaks me out. What I once thought was my complicated little unique snowflake of a brain turns out to just be the combination of 21 values.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by Goladus on Monday April 02, 2007 @10:20AM
(#8206)
I am fairly certain that telling me the test was going to compare me with Y Combinator startup founders affected what was going on in my brain when I decided what images to pick. I'd do the test differently under different conditions.
Re:Pairwise and Y Combinator: Do you have what it takes to b...
by plusbryan on Friday March 30, 2007 @09:20PM
(#7745)
Thanks for all the comments, good and bad. They help us refine our test. Keep in mind that we're attempting to figure you out in 40 pairs of pictures. We did this so that lots of people could take it, just to introduce the idea.
A real test would have more like 200 pairs, which is what the YC founders took when we assessed their attributes in the first place.
Now, this is fun stuff to burn some minutes while procrastinating! Here are my results:
http://www.likebetter.com/quiz/results?quiz_id=1125255845&user_list_id=6
Some things that the test got backwards:
- I'm more a listener than a talker,
- I never smoked (I clicked on the hemp pic because I like intense green :),
- I think I'm more into books than my score there suggests,
- I like mild food rather than spicy,
- I do like and have pets,
- I'm not really that urban. I'm more comfortable in villages or small towns.
Interesting. I posted a similar post this week on the characteristics found in entrepreneurs ~ http://news.ycombinator.com/comments?id=7459
'... all we'll care about is one measure: how close one comes to the best founders. We don't care what atoms are in that molecule ...'
But testing for characteristics of 'high achievers' then correlating test scores against potential applicants is fraught with problems. Is the correlation against the best enough? [1]
I'm sure you might get some correlation but it's pretty unimaginative [2]. So what could be a better tool? What about direct testing of skill? Wouldn't a better technique be simulation? Why not build a 'startup simulator' where applicants are given tools to simulate tasks they actually will have to do? Then you have a controllable scenario where you really can measure results against successful founders.
You could test
- the conception of an idea
- the building of a (simple) prototype
- quickly find an audience
- find a way to make money off it
Wrapped up in a framework [3] where you test the execution of these tasks you could get a better grasp of the
- skills
- determination
- entrepreneurial audacity
- passion
- humour
- leadership skills needed to succeed.
- risk taking, emotional intelligence
There is a long history of simulators in testing & refining of skills, competency and execution are required. Even Captain Kirk at Starfleet Academy trained, passed (and cheated) on simulators.
Who knows it might even be fun.
Reference
[0] Professor Con Stough, Brain Sciences Institute, 'Director, Centre for Neuropsychology, Swinburne Institute of Technology'
http://www.swin.edu.au/bioscieleceng/neuropsych/stough.htm
[1] But I do like the simplicity of just looking at a simple set of parameters to make a Gladwellian decision.
[2] In a competitive environment the difference between getting the attention of the right candidates could be the tools used to measure how they stack up against the best .. but actually doing something, learning and getting some feedback.
[3] The framework could be a game, or simply a panel of dials. The key thing is you can have a back room with founders, past entrepreneurs twiddling the dials checking to see in RT how candidates handle things like a person leaving, stressing system etc (like the LEM tester in the Apollo missions). It also allows for candidates to learn from failure with less risk & pass on the best decision making skills to candidates that have passed.
I mean before Y combinator, build a product - launch it. Grow the userbase - iterate.
Reality is the superior test of whther you are an entrepreneur - don't need to get Y com funding to get going...
See all the recent fundees: Scribd, Boso, Weebly, Buxfer: they had a product.
Doesn;t need to be so far down this track but having done something is better than simulating it.
True . Ideally yes. But it there may be times when you want to test skills or sub sections of skills, NOW. I've run 8 miles in 2 hours in the past but does that mean I can do it now?
I think the bit I was addressing was suggesting a technique to scale selection through automated means by testing against the best. I agree reality is a better test. But you miss the point about simulation. With simulation you can
- test/be tested/train in compressed time
- test/be tested/train with different scenarios
- test/be tested/train individual components of the process
- test/be tested/train with lower costs
- test/be tested/train in controlled circumstances to allow better measurement
'... having done something is better than simulating it ...'
Pilots fly planes but they also train and test their skills in simulation due to cost, risk and a lot of other factors. People who do things, need ways to test their skills, improve and be tested. Why not startups? In the case of YC, 'done', 'doing' may not be controlled enough to measure against the best (conjecture).
One thing I wonder is whether there is any logic to deal with confounding variables--for instance, if someone picks every photo that has a bike in it, maybe they just like bikes, and those data points should be excluded.
On the whole, though, the pairwise people have probably as good a shot at anyone at making a not-suck personality analyzer.
Why don't you just use a classic IQ test if you are into that kind of stuff, or the Myer-Briggs test?
http://www.likebetter.com/quiz/results?quiz_id=1449411178&user_list_id=6
I think the attributes of the test, should be more questions of type, what you do in specific situations, and be compared to what YC would prefer as answer.
update
or am I wrong, and GREEN is the range of answers of the samples?
The premise that pictures and the choice of pictures determine anything about yourself is simply silly. I have different experiences than someone else. I have been to and seen different places than someone else. Different imagery will provoke different emotion in different people.
I found it weird that Y Combinator founders generally fell in the "aren't much into books" category.
Here's Mine: http://www.likebetter.com/quiz/results?quiz_id=509484655&user_list_id=6
Seriously, what impact does this have on getting funded by YC? (I'm sure everyone wants to know the answer to that)
In particular, we don't care much about the specific metrics this test tries to measure. The Pairwise guys chose those; we had nothing do to with it. When we use this for real, all we'll care about is one measure: how close one comes to the best founders. We don't care what atoms are in that molecule.
And so the test will need to get more sophisticated in turn by finding even more non-obvious yet discriminating pairs of images.
This is sort of like the constant battle between those creating spam filters and spammers.
online dating
airport security
immigration
finding cofounders
You might as well hire a psychic to test potential cofounders... I think you'll have the same success rate.
Does anyone remember The Spark's gender test? It's a specific case of this kind of test: http://community.sparknotes.com/gender/ (use bugmenot.com for login)
http://www.likebetter.com/quiz/results?quiz_id=1627867125&user_list_id=6
I am too depressed to read any further:) But I did - it only gets worse:
"read books voraciously"...I'm as far from that as you can get.
"a religious person"...tell that to my mom
"have or want pets"...can't stand most pets!
http://www.likebetter.com/quiz/results?quiz_id=918442805&user_list_id=6
And it knew that. That's really strange and freaks me out. What I once thought was my complicated little unique snowflake of a brain turns out to just be the combination of 21 values.
http://en.wikipedia.org/wiki/Brain_fingerprinting
http://www.likebetter.com/quiz/results?quiz_id=596593159&user_list_id=6
So is it good to be on all of the edges?
http://www.likebetter.com/quiz/results?quiz_id=556637953&user_list_id=6
A real test would have more like 200 pairs, which is what the YC founders took when we assessed their attributes in the first place.