These results may be "curved".
If you give the same easy test with no pressure and everyone is sufficiently good, then it was a waste of everyone's time.
In such an approach, you do something that is sufficiently hard that you're able to identify the people who are the better performing ones along with a reasonable idea of how well they work under pressure (because that will happen at some point).
Is it testing the right thing? Probably not. However, we still haven't found "the right thing" that is able to scale, respectful of the time of the candidate, not too intensive on the interviewer, and minimizes biases in hiring (same set of questions to all candidates, same grading scale).
Maybe because the concept of a test is flawed? It seems to me like every company that uses a whiteboard test gives out the same tests anyways.
What are they really trying to test? If someone understands common algorithms and data structures? I dont see why that couldnt be determined by a simple interview with an engineer. A test might be useful for a HR recruiter who doesnt have a computer science background, but those tests are usually conducted by other engineers anyways.
The whiteboard thing just seems to me like an arbitrary hazing ritual.
This is more a random fluke as much as sampling bias. Akin to someone either getting lucky or someone who is coming from a very similar environment. This isn’t a measurement that guarantees someone is competent, rather it’s a guarantee that someone is coming from just as incompetent an environment as you have.
Speaking only of ICs:
From principal engineers down, there's a technical screening. Distinguished engineers? Maybe. But then you're speaking about Guido van Rossum, James Gosling types.
My experience has been that this applies to recruiting, hiring, BD, sales, fundraising, etc. Anywhere that there's a systemic principal-agent problem.
I've heard that advice in various forms but I must say (anecdote incoming) - in my own 30 year career, the best jobs I've ever had were the random ones that I just applied for and the worst ones were the ones that somebody I knew from before recruited me in from outside.
I am wondering if the entire interview process should just be pay them for 2 weeks and see if they stick.
Am I on the top of my game right then and not stressed about anything and the questions out of the random million possible questions that could come up are ones that I just happen to have had on my mind recently? Then I ace the interview.
The last time that happened it was in a big international consulting company, they then put me through 5 other interviews afterwards, each one telling me I would hear something back in the next couple days.
The last one I got told couple more days, they didn't call me back for nearly two weeks. Then they call: Hey Bryan, we would complete the offer process now hope you're still interested?
Uhm unfortunately I got an offer last week for a 6 month consulting project, sorry but I had to take it because they said they couldn't wait to see how this turned out.
on edit: clarification.