Hi guys, this is Simone Brunozzi, the "new hire" at Amazon.com. Second Life is not the new holy grail, I just said that it was a key factor for me. Second Life is still in its infancy, and it's useful only in specific situations. I consider evangelists very important for an organization such as Amazon. They don't have an explicit position on Second Life, though. Best,

Hi, one thing that I might take issue with is this:

> poorly developed nation from a technological point of view.

There are tons of bright hackers in Italy, in my experience, and in various fields, there is good tech that comes out of Italy. The problems are political and economical in nature, in my opinion - the technology is there, it's just not utilized well because of all the crap that happens politically. As one small example, it between "capital requirements", and all the fees, it probably costs something like 10000 Euro to create an SrL (limited liability company), whereas in my home state of Oregon, I managed to create one for a little over 100 dollars. I'm sure you know all this, but many people don't realize how many smart, entrepreneurial people there are in Italy, and only see the results of years of bad policy decisions.

Complimenti, in ogni caso:-) Leaving Perugia wouldn't be an easy decision for anyone - I think that area is one of the most beautiful that I've ever seen, anywhere.

Is it just me, or does anyone else find 14 interviews (11 on site, 3 phone; one more if you count the fair) a bit ridiculous? I mean I know people are the most important thing and you have to make smart choices but damn!

Yes, after three or so I would start saying "look, are you interested or not?". Time is money, after all - perhaps one should simply ask for money for further interviews?

Yes.

One the one hand, companies like Amazon whine (to whichever politician is listening) that there's a desperate shortage of skilled technical people. On the other hand, they subject new hires to 14 interviews.

Tell me: is this the sort of thing you do when you're in desperate need of employees?

Are you proposing that they lower the standards?

What standards? This is just geek chest-thumping, and has little mathematical justification.

Let's assume that your interviewers are uniformly exceptionally good, and have a false-negative rate of only 10% (i.e. 90% of all good applicants are accepted).

An interview with a good candidate is therefore a Bernoulli trial, and the outcome of an interview loop can be modeled exactly as a binomial process. If you conduct 5 interviews, the probability of at least one interviewer saying no to a good candidate is 0.41. If you conduct fourteen interviews, the probability of at least one false negative is 0.77, and the probability of at least two false negatives is 0.42. In other words...you've made it twice as likely that you're going to have someone falsely reject a good candidate.

Meanwhile, let's assume that these unrealistically brilliant interviewers are even better at rejecting bad applicants: they each have a false-positive rate of 5% (and therefore, a true negative rate of 95%). If we assume a five-interviewer loop, the chance of a single false positive is 0.23. But bumping the number of interviewers to fourteen raises the chance of a false-positive to 0.51! With fourteen interviewers, the chance that one will say "Hire" to a bad applicant is over 50%!

In other words, with every interviewer you add to a loop, you raise the chances of a false-positive or (even more likely) a false negative decision. If you have normal interviewers (i.e. people who don't have super-human discriminatory ability), the problem is even more pronounced.

Problem - 1 false negative or 1 false positive isn't usually a problem. Generally, if you have this many interviews you have some sort of voting system. Let's say it's consensus - then 1 false negative will disqualify some good people, but you'd need 14 false positives, you have a (0.1)^14 chance of hiring a bad person. Expected number of interviews in which you would see this happen once? 10^14

But, you may ask, what about the case where they should be rejected, 8-6 (assuming majority voting not consensus, but you could apply this to any voting threshold), and you have two false positives and 0 false negatives, is it really that horrible that you might experience the borderline case where you hire someone who might be mediocre, assuming everyone else is pretty good?

Yeah, there is a good chance you'll makes some individual mistakes. But having lots of interviews decreases the probability of making those mistakes frequently enough to actually influence the outcome.

Hate to say it, but your "mathematical justification" is really just another example of geek chest-thumping.

Well, obviously, you can keep your false-positive rate low by requiring absolute consensus. If you require fourteen different people to agree on every hire, then you're not likely to hire any bad people....you're also not likely to hire anyone at all.

Your second argument is more reasonable: if you stick to strict majority-rule voting, it's true that you're less likely to get a false-negative result with a larger number of voters. (You'll also increase the number of false-positive results). Problem is, nobody actually does it that way. Most hiring managers are trying to keep the false-positive rate low, and tending toward consensus. Thus, random false-negatives have quite a bit of impact -- one or two "no-hire" votes are enough to torpedo a candidate at places like Amazon, Microsoft and Google.

Clearly, when you do enough interviews, you start to expect "no-hire" votes, just due to chance. You can claim that this is good for the company, but like I said before...these companies do a lot of complaining that they can't find "qualified" applicants. Perhaps if they re-thought screening processes that involved dozens of interviews, they might find that the qualified applicants were there all along.

This is why I love math.

Yes. 12 interviews should be plenty.

"Lowering standards" doesn't work as a sound bite, but if something is too high, one lower it, even if it is a standard. I don't think standards are sacred.

For instance, if you used to be a very desirable mate and had standards to match, but are now less desirable, you should lower your standards accordingly.

I don't think the issue is about lowering their standards, but rather re-evaluating them. Throwing more interviews at applicants doesn't give you much more info beyond a certain point (which is presumably somewhere before 14).

To be completely honest... that lost a ton of integrity as soon as I read "I showed up at the Amazon.com’s booth, with my well-dressed and custom-skinned avatar which, as a bonus, has my same name, Simone Brunozzi (don’t ask me how I got it because it’s a secret)."

Cmon now... what is this virtual ^&*#$@! coming to? Really Amazon.com? Really??

Edit: Not only that... but look at the view from his office... I'm practically speachless at how BS not only that job position and article, but Amazon.com's participation and movement in that direction is...

Does it really matter if they find the right candidate through Monster.com, on SL or while they're shopping at Costco?

Nope, not at all. The reason why I am slightly offended is that people are treating "virtual interviews" with the same seriousness as traditional, face-to-face interviews. There will be no substitute for real human, spoken, face-to-face interviews; more to take out of a person like body language and emotion. Human collaboration within a close environment is crucial to most aspects of business and production. It is nothing more than silly for an individual to feel accomplishment from a job interview via SecondLife; feeling successful, bypassing face-to-face interaction. Though, I feel this position might seem too harsh.

Where does it treat them with the same seriousness?

To me the initial SL component was just a screening / outreach process - finding them someone they wouldn't have otherwise found.

He then had 11 face-to-face interviews (in two different countries) totalling 9.5 hours of face-to-face time..

How was this "bypassing face-to-face interaction"?

What does a virtual job fair have to do with integrity? It let an HR person from Seattle interview someone in Italy for a job in Luxembourg...

To me it seems like Amazon being smart.

Regarding your edit, I don't think you understand how important technical evangelists are at places like Amazon, Microsoft, Sun, etc... They are essentially technical sales-people; they're the ones who say "Hey, this is what we're working on, and this is why it's important".

To be completely honest with you, your post seems like sour grapes.

Interest is a two way street. It's on the job candidate to be qualified for the job they apply for and to not waste the companies time if they're not interested.

On the flip side, companies need to be respectful of a candidate's time and make a move if interested or let them know early on if they're not so the prospect can move on.

I work for Sun. I had one interview, WITH my manager. She made the call, I got the job. I think AWS is a great service offering but it sounds like they may need to fine tune their hiring pipeline a bit.

I do a lot of, for lack of a better term, "geeky" things, but I would never write about anything involving Second Life. Perhaps it's unfair discrimination and perhaps I am just ignorant to SL's real worth/usefulness, but... I can live with that.

I disagree with you on your edit, however; "evangelists" can be very important to an organization.

Bravo. Well done with the interviews and the tips to new grads and interviewees. Perseverence and studied methods to distinguish oneself from the crowds is the way to land any job. I'm glad he found a job of his liking in a great location.

I'm guessing that's not going to be the best job ever. Maybe it's Second Life stigma, but I don't think so.

Actually, it will be the best job ever. Technical Evangelism is impactful, fun, and all-consuming and I would be happy to tell you more off-line.

As Simone's new manager I am happy that we were able to find such a talented person using Second Life as our first point of contact.I am also happy to know that he has experience and interest in leading-edge technologies.

Hiring great people is difficult, time consuming, and expensive. You have to find them, screen the resumes, conduct phone screens, and ultimately meet them in person. If there's a way to use an advanced technology to optimize and reduce the time and expense involved, then what's the problem?

"I would be happy to tell you more off-line"

I would be happy to hear more off-line. My email address in is my profile.

Sometimes it takes someone's ESL to get the best bulletin board material:

"if you work hard for something, in the moment you stand in front of your challenges you’ll feel stronger than ever"

Bravo!

I'd wish you luck, Simone, but no need. Looks like you already know how to make your own.