What I've learned is that it is a very weak signal to talk to candidate about "their research" unless it's immediately and directly applicable to what we were doing as a company, so that we can talk about it as equal experts, which never really happened in our case. Previous research done by candidate was, like research almost always is in a narrow and specialize niche. Every candidate I was talking to was very good with talking about their research and from my perspective at the time of the interview it was impossible to validate and evaluate their claims. That conversation could convey the character of the candidate but I couldn't evaluate their ability to do new research.

Instead, presenting candidates with the same problem, that in our case took around 3 hours to solve, in our case proved a good way to compare candidates a bit more objectively and test how they approach solving problems they haven't seen before which is more applicable to what they'll be doing on the job.

The downside of that process is that during three hours you can only test so much, but hiring is a really hard thing to do right and we didn't want to take more of candidates time as we did value it highly and found that to be the sweet spot for us.

It all depends on the questions asked to the candidate. You don't ask them about their research which is publicly available in any case. You ask them questions that let them reveal their thought process about their research. Now, it's up to the interviewer to come up with good questions to make this happen. The interviewer has to be well prepared and think about questions. You don't need to be an expert to ask good questions.

I'd like to reiterate that this applies to research/scientific position where pushing the boundaries is a crucial aspect of the job. In case you are looking for a very specific skillset, things may be different.

You had a lucky escape. I had the same from a company. Which was arguably a 2 full nights worth work if done right.

I took the challenge. Did it, it passed their functional test suite which they provided. Unit test cases, git repo, documentation etc etc all in place.

They came back saying it was not enough. So it's not like if you do the take home project, they let you in.

Also its a tad little irritating to make people work for so long, make them do everything. And even then with perfectly working solution turn them down.

Further inquires about what was wrong, given that it was working as per specification got zero answers.

Anyway, this all seemed perfectly reasonable until I went for a job I'd been approached about by an agency with a similar test at the beginning of the application process. They'd asked me to build a simple calculator and said it should take no more than 2 hours.

No big deal, but it was the weekend before I had 2 hours free where I could do it due to other commitments. Pushy recruiter didn't understand this but, whatever[1]. Should have been a red flag straight away.

Anyway, I put together a working solution with comprehensive tests, send it off to the recruiter and on Monday hear... nothing. That's right: nothing. Gave it a few days, got in touch with the recruiter and still nothing. No, "sorry, it wasn't good enough." No feedback on what might have been wrong with it. Nothing.

I was absolutely incensed. The whole incident gave me a different perspective on pre-interview tests, to the point where I'm no longer really interested in doing them for "unknown" companies. I made an exception to work with an organisation where I had several friends, so was confident that even if I didn't pass I'd get some decent feedback on why.

The most significant effect, however, has been on my own approach to recruitment. At the moment we don't, and I have no plans to introduce, programming homework as a pre-screener. Rather we take a much more personalised approach. We're also in the situation where we get few enough applicants that we can afford to talk to all of them, although this won't scale and will have to change.

Still, it's hopefully uncontroversial that before you invest the 5-6 hours (across 2 people) required for a 2 hour technical interview (in our case an exercise where we work together on a problem), you do need to some sort of pre-screener to weed out the people who really can't code: https://blog.codinghorror.com/why-cant-programmers-program/. I wouldn't say this is 99 out of 100 programmers, nor anything like, but it's certainly the majority. We therefore do a half hour telescreen, which is a fairly informal chat about what the candidate has been working on, what we do, and then - of course - there's a very simple programming task at the end. This takes an investment of roughly 90 minutes on our side (again, 2 people). We're explain the format to candidates in the invitation, which gives them an opportunity to decide whether they want to go through with it or not.

Even if candidates don't pass, and it's usually on the technical task that they fall down, we give them feedback, including pointers to how they can improve their skill level.

(Apologies, this came out much longer than I intended.)

[1] There's some merit to the "well, do you want a new job or not?" mode of thought but not much. E.g., in my current role I advise candidates not to do our technical interview after a full day of work because they're not likely to perform at their best. Obviously not always possible, but it's about ensuring they set themselves up for success: if you dislike your current job enough to be looking for another one don't you want to make sure that when you get an interview you do everything you can to do well in that interview? This is obviously very different to expecting candidates to jump through hoops under time pressure just because "commitment".

What too many people choose to forget or ignore is employment itself (at least in CA) is a 2-way process. The employer is not doing you a favor by employing you, despite how many people have drank the kool-aid to the contrary. You and the company have entered into a mutually beneficial agreement. Unfortunately, many people don't have the means or desire to walk away when an employer veers into "you should be thankful we let you work here" territory.

Why are software engineers doing interviews at all? What makes software engineers qualified to interview people (especially when you consider the need to avoid hiring bias, problematic statements by interviewers that expose the company to legal suit, etc.)?

I'm not saying that companies should go to the other extreme and have all interviews conducted by HR - that presents its own set of problems in that HR doesn't know how to evaluate candidates for technical skill. But expecting hiring managers to open up a few evenings per week in the irregular and relatively uncommon circumstance (if it's common, then you have problems with churn on that team, and you should consider firing the manager) in which positions open up on their team to evaluate potential direct reports is not even remotely unreasonable.

i do want the opinions of the people already on my team, because a big part of the interview process is to find out how well they work together. for me hiring is a team decision, not a top down mandate. so yes, all of my software engineers will be doing interviews. there is no way around it.

on the other hand i agree with you. these rounds of interviews should not happen to often. if our team grows, i expect to add two more positions every quarter. (i don't know how realistic this is, but this just to give you an idea of how much interviewing workload i expect.)

there is only so many days i can take off. with two weeks holidays, i might spend all of my holiday budget for this year.

by stating that it is most certainly not just once as i am likely interviewing for multiple jobs.

i don't know what the solution is, but i think it is reasonable to say that some compromise is necessary, and as much as i value your (and my) time with family, if we want to grow our team so that we are not having to work overtime as much and actually get more time for our families, then we'll just have to make some sacrifices.

apologies for being a bit polemic.

actually, i do have a solution for myself. i only hire young people fresh out of university who don't have a job yet. higher positions are filled from within. ask me in a few years how well that's going.

I'm honestly not sure it is. Half-a-day to a day of on-site interviews (possibly after a phone screen) is how most companies have been filling professional roles for decades. Certainly it's an easier process for people doing it out of university. And it's easier to do locally when a candidate can often just take a few hours off. But, while many debate some of the specifics of how companies screen for technical talent, I don't see widespread pushback against candidates coming in for on-site interviews of some sort.

That being said, if a candidate just couldn't take off work for whatever reason, I am willing to do an evening interview.

Gotta be flexible, on both sides.

It seems to me this is an issue with your employer not the interview process.

I'm currently interviewing and so far I have had a 48 hour weekend project, a 3 hour code assessment, and I'm expected to endure 3 days of interviews. I'm fairly close to throwing in the towel and saying I can't be arsed with this.

This may not apply to all companies or situations, but for a Microsoft-sized organization this is the way it should be done.

Your hiring needs should reflect the size of your organization, and your hiring process should reflect your hiring needs. If you hire relatively infrequently then it makes sense to go over individual resumes and have infrequent evening interviews. If you need to hire hundreds of people this month just to negate churn then you have the resources to build out the kind of enforcement-process-oriented above since you surely need that kind of infrastructure anyway. Everything in between is just a matter of degree and where your company is on the process-vs-culture balance as a function of its size.

However TBH an in-person day with 4 or so interviews (perhaps post a phone screen) is absolutely standard for all sorts of professional positions and has been for many decades. I can only think of one case where I just had an informal in-person lunch instead--and that was a case where I knew the company's owner very well.

Any serious interviews will take a day extra traveling, your prep time etc - I agree about having 4-6 interviews in a day is just stupid.