It'd be nice to think I have some special skill here but I really don't. This is just how interviewing was done in the 90s. To some of the younger generations, I've been told it sounds crazy.
If you send me your resume I'll actually read it, carefully. If the person described in this resume fits the background experience the role needs, you get an interview.
During the interview we'll talk about all those projects you worked on that are relevant to this role. Which parts you enjoyed the best and why? Which parts were boring and why? Which parts were the most challenging and why? What you find too easy and why? What would you have done differently? Could you have? If you were to do the same project all over how would you approach it? Other open ended conversations along these lines.
I don't ask anyone to whiteboard code, that's not part of the job so it's not part of the interview. No puzzles, no trivia-pursuit style questions.
It works great. You can't BS your way through such a conversation with a senior technical peer if you didn't actually do the work described in the resume. You just can't.
It is, however, vital that the interviewer must be a expert in the field.
> > No puzzles, no trivia-pursuit style questions.
I swear HN technical interview threads are the poster child of talking past one another. First, for-loop is nowhere near a trivia-pursuit question. Second, different companies of different sizes/industries/goals have different requirements. Let's all move forward with this discussion and acknowledge that we can't all use the same process because we're not all hiring for the same type of job. If the engineers you hire consider for-loops a "puzzle" that's totally fine and OP using it doesn't invalidate your process for your multi-national or startup companies.
> I can say that I've never regretted a hire I said yes to
The real question is, has anyone else regretted that hire, which unfortunately can't be answered as they may not tell you.
> To some of the younger generations, I've been told it sounds crazy.
Doesn't sounds crazy at all. A single process guaranteed to work for everyone? Now that sounds crazy.
1. It's often surprisingly difficult to fire an underperformer (anecdata: a friend of mine has lost multiple talented members of his team in the past few months - all the result of having to work with one completely unqualified person. For some reason, the other person has not been fired). As a result, a bad hire can have an outsized and lasting effect on a company.
2. Unless you're looking at very senior positions, there are almost always more people available to fill a role. You may miss 9 out of 10 people who would be effective in the role, but you only need to hire one person.
The math changes a bit when you look at the mythical 10x developer, so maybe it would be worth looking into false negatives specifically in that case. Still, getting that 10x dev would be much more important in a more senior position.
You don’t want ‘bozo cliques’ to form, so you make a semi-objective process like ‘solve this algorithmic question’ as part of the interview loop. I think execs and the founders doing a final review before hiring all engineers comes from that fear.
Then other companies cargo cult interview processes from larger companies and the trend propagates.
If you want to ‘hack hiring’ as a smaller company, you should use hard to scale processes like the one described in the parent post.
I love this idea but my old history professor (who I pretty much disagree with on everything except this) used to always say "follow the money". What are the incentives that all the parties have for and against growing companies bigger?
If you don't actually verify the technical problem-solving ability of the candidate in some way you're forgoing signal that can massively increase the confidence you can have in your decision.
After a few bad interviews I got the hang of it and aced a couple algos interviews. I’ve worked as a developer for 8 years, have tons of software in production, have some open source contributions and have worked productively on several teams.
Interviewing is a skill. I understand why a company wouldn’t hire someone who doesn’t pass a programming test but failing a programming test doesn’t mean you can’t do the job.
Then ask for a reschedule or try to get it as the first thing in a day.
> I found the interview process stressful, or felt nervous or uncomfortable in the moment
Need to grow confidence. Walk into interview - like you own it. You are the best. Think - you know everything {only during interview :-) } but also keep an open mind. Don't think - it's an interview - take it as you are going to teach them about what they will ask.
Irrespective of this - mistakes will happen. Fail -> learn -> try again.
Yes, I wrote that right. He struggled to understand conditionals in general when building his own logic. The guy even had a masters degree.
I then did a course on assembly programming and having to be extremely structured and using jump statements actually helped me a lot in easier to use programming languages.
Point of this comment is that people might be good at a lot of things, while just having some weird brainfuse related to a very specific thing. That said, not using if statements makes programming rather difficult.
What did he do or say that made you think he could do the job even though it looks like he wasn't able to work with other people code?
But they blank at code and struggle with the basics. They get lost for hours at the most trivial bugs. It's weird, but it's very real. Real coding takes a certain kind of abstract thinking that some people just don't have.
A company trying to hire engineers can easily give a take-home programming challenge to a dozen engineer applicants and take very little time analyzing the submissions. That’s pretty unfair. But it also feels a bit untenable for a senior engineer at a hiring company to have very deep investigations of each applicant’s work history.
Another problem is objectivity. If your company’s engineering hiring process relies heavily on a senior engineer’s subjective impression of an applicant, you’re going to have big problems with your own engineers’ biases, whether subconscious or not. Expect to hear a lot of evaluations like “well, the applicant did seem to have good knowledge and experience, but I just wasn’t impressed for some reason.”
Is it? The cost of hiring the wrong person can be huge. Not just agency fees if they came through a recruiter (those aren't cheap!), but also all the time people then spend on the bad employee and all the damage that person does before the mistake is rectified.
If this system of reading the resumes and then spending a few hours of senior employee time on an interview can reliably find good employees, that's an absolute bargain.
This just isn’t true. Every company has a “probation period” usually 3 months where either party can terminate the agreement. That’s more than sufficient to cover this risk.
You don’t pay the recruiter until probation is passed - everyone knows this.
This meme comes from Spolsky who somehow also convinced the world that the hottest programming talent was beating down his door to work on a bug tracking tool for project managers. Maybe his talent was in blogging, and not actually in hiring?
Sure it is. People on our team need to take time to onboard the new hire. That's good and expected. The new hire will work slower and that's ok; they will need extra help, time to learn the codebases, etc. After a couple of months, a particular new hire did not work out. Time had to be taken to document reasons, meet with HR, meeting to talk about expectations, etc. In the end, the new hire is gone and so is the time the team spent helping, and the slowdowns on real deliverables, and the hit to team morale. It sucks when someone is let go. It was a net negative for our team and thus the whole org. Depending on a combination of level (leadership position?) and toxicity, negative impact on the team or company culture can occur. Bad hires can have a real cost.
I believe that to be an incorrect statement, based on my own experience. I have seen companies hire the wrong people and pay the price.
Every company has a “probation period” usually 3 months where either party can terminate the agreement.
I believe that to be an incorrect statement; while I've always seen probationary periods, in this very set of threads "akelly" says that nobody has probation periods. This evidence suggests that "akelly" has never worked at a company with a probationary period (and has foolishly extended their own personal experience into the universal, but that's a separate mistake).
I would also suggest that there is an opportunity cost attached; if someone is fired at the end of their three or six month probation period, the good candidate that replaces them is six months behind.
Some companies end up with drifting dead wood employees; not fired, just moved around from team to team, department to department, because the company makes firing people harder (or more costly to the manager / team-lead) than moving them. This can go on for years. That seems quite a large cost. I've also seen managers simply sideline bad employees rather than dismissing them, for reasons of company politics and face. I've also seen bad hires get promoted, in cases where promoting someone is easier and less costly (to the relevant team lead or manager) than dismissing them; that can be really damaging.
Yeaaaah no. I've seen people nope out of a job within anywhere between 30 mins to a couple of months. I've also seen people that knew it wasn't right for them stay 1 to 2 years.
I have seen not that brilliant employees that still were earning company money. Just you know "we hire only best of the best" is a scam.
Hiring bad manager or a senior dev, can be bad I agree. So it is not so good to use the same hiring way for seniors and juniors, but unfortunately companies treat senior devs also like juniors.
I agree with your approach and use it myself but one thing is different now and that’s the proliferation of tiny skills. Back then you would have a few big skills, you would claim to know one or two main languages, one or two databases and so on. Now people list hundreds - literally hundreds - of skills sometimes. And there’s no way to tell on reading if they really know it, or just saw it once and maybe did a tutorial or “hello world”. People now will add a skill to their CV if they’ve done it for a hour total in their entire lives! Or read a blog post about it. It is a massive time sink to pick through that.
This tells me the person doesn't know the difference between skills and implementation details, so I don't need to proceed to an interview.
Effectively you penalise not catering the candidate resume to what you are looking for or deem important.
One will filter you out for having too many keywords. One will filter you out for not having enough keywords. One will filter you out for including a picture. One will filter you out for not including a picture. One will ... ad infinitum
Your resume is being evaluated by different filters constantly. Different filters filter different things. There is no "right" answer. There are only answers you will or won't be filtered out for depending on which filter is being applied.
Which filter is being applied?
It's random.
Hiring is essentially random.
"I need my resume to show up in your filter, relevance to the actual job functions be damned."
I know the theory of how to use it on large projects. I even know enough to know that I'm pretty much just scratching the surface, but so are probably most other people.
Same with most other skills. At how many lines of code can I call myself proficient in a language? And what if I copied large parts of code from stackoverflow?
All in all I think I spend like 8 hours on writing and formating that freaking thing, and it still just feels like some of my skills are a stretch.
> At how many lines of code can I call myself proficient in a language?
If you want a target to shoot for, I'll give you one, but don't really follow it... LoC is kind of meaningless on its own in part because it's contextual on many things (you highlighted one context, lines copied, I'll use a different context). 1k lines if paid for them, 6k-10k if not. This doesn't include lines added and then deleted (i.e. if your only project in FooLang weighs 300 lines, it's 300 lines, even if making the project involved adding 100 lines, deleting 50, adding 500 lines, deleting 100, editing 300, deleting ...) you need projects whose sum mass is 1k. Probably better to have at least one 1k+ project too, especially if it's unpaid. But again, don't follow this, it's just a somewhat random target to answer the question. Better to think about what this could be a proxy for, and target that instead.
I'm glad you brought up git. Source control is central to modern software development. If you don't have it on your resume, someone might think "hmm, have they really done much programming?" But if you do put it, it will obvious from your resume that you are just out of school, so no one is going to expect you to teach a git class.
As a more experienced person, I won't expect you know the internals of git.
Non-technical people screening your resume aren't going to know the difference between you and Linus, though, so you need to put it. Don't worry, it's not false advertising.
If the job description is back-end tooling at Github or Gitlab, I'll expect more. Similar to the difference between knowing how to use a spreadsheet, and having written a spreadsheet.
Now, if it's something crucial to the job,say your C skills writing software for micro-controllers at an embedded electronics company (like what Nest or Pebble used to be). You can't fudge here. Nothing will help but lots of practice.
Good luck!
P.S. For your first job, focus more on figuring out what you want from your working life. What kind of manager, what kind of work, etc. Yeah, you'll need to pick up some buzzwords to add to your resume, but it's not as important as becoming good at the fundamentals of what you are doing.
To be honest, the process of writing a CV is mostly painful for me because I have to evaluate myself and feel quite inadequate. At the same time, I have multiple companies wanting me despite not being very good, so my actual chances are really good. Software development in Germany is going crazy at the moment.
I would say don't use adjectives to describe skill level. I put most languages I have experience with on my resume, and order them descending by how well I know them. I know the stuff at the top pretty well, and it's downhill from there. In interviews I'm very upfront that I'm incredibly rusty with the stuff on the bottom of the list, and interviewers understand and accept that.
The exception is if you feel comfortable putting "advanced" or "expert" on one of those, but then you had better be able to back it up.
LOC is a bad measure of proficient. I would say you're proficient if you can write a simple program without looking up syntax and relying only on autocomplete for library references.
It takes a lot more effort on the part of the interviewer and is "harder to scale", in that you can't just train people to ask canned questions. But since it's open ended, it's a lot better at finding out what the candidate is really good at, and it's very useful to have one of these in every hiring loop, usually by a senior team member or exec.
This is key. There are a lot of hiring managers masquerading as experts and are frustrated when cargo culting hiring processes falter and lack the people skills to diagnose a situation. You'd also be amazed at the quality of resumes a high, advertised salary will bring.
This is so true. The best devs are making above market rate and are busy and content with their current role. By not including salary most of these people will simply ignore you, because of the time risk involved in finding the number. Non disclosure only works if your salary ranges are generally known (eg faang).
I don't disagree that there are a lot of fake it till you make it developers who did a vo-tech class and are applying for jobs out of their league, but I just don't know how people don't spot them. It takes me about 5 phone calls to find a competent individual, and then I bring them in for a in person. As a hiring manager, I don't find it cumbersome to weed thru 5-10 people to find a good hire.
You think this is a claim about how good your hiring practice is, but the only way I can think to read this is as a point about how little hiring you do or how little evaluation of hires you do. In the real world, perfection isn't possible, so claims of it are a sign of inexperience or naivete.
This may just mean that you say a lot Of wrong “No”s. To get very high precision or very high recall is really easy... what you must measure is your F-score
Remember Facebook rejected Brian Acton, and paid billions for that. An oversimplification, I agree, but the larger point is that people tend to dismiss too easily the impact of bad “no hire” decisions, compared to the bad “hire” decisions. Just because you can’t or don’t measure it, doesn’t mean that the impact may not be arbitrarily large.
To borrow from poker: it’s commonly understood that if you’re not “caught bluffing” at least a little, it means you’re not bluffing enough. If your hiring process results in zero bad hires, you are playing it way too safe and I guarantee you are missing out on phenomenal candidates.
Now, this assumes that you're not missing out on great hires; in other words, it assumes that the good hires you miss are at the bottom of your hiring range. Your point might be that this can't be guaranteed, and while that may be true, it's unlikely that any tweaks to a given hiring process are going to bring them in either.
I would be honest and say "I'm under NDA for a lot of that but I'll do my best". If neither of you are jerks, you should be able to manage it.
Have you let new hires go in the first 90 days? That's just one of the ways personal regret is probably not the best metric here.
This is one way the current status quo might be better than the past: you don't get pigeonholed so much by your past experience into being a "fit" only for similar roles. Sometimes the hiring manager is really looking for a specialist, but in general, we don't care what industry you were in or what tools you were using, as long as you can prove you're smart. Some of the most impressive people we have working on Go microservices were enterprise C# developers before.
While it maybe easy to hire dev who have been on the market for 10 years. You still have to keep in mind that new developers are still a very large proportion of the dev population.
If someone can keep up in a technical conversation about their background with me and answer every question I have about a technical project they did, then basically they pass. It works especially well even if I'm not familiar with their project, because I have an opportunity to learn, so I can ask any question that comes to mind until they teach me what they learned.
I did hire someone that I regretted, though, but to be fair, this was among my first interviews. The mistake I made was getting too easily caught talking about programming and technical things without specifically diving deep into his past project. He and I vibed quickly and I liked him, and that felt like enough, but after only a week it seemed obvious that he wasn't going to be producing much code, and we let him go. Otherwise, I've been happy and my ability to discern has only gotten better as I became more experienced.
I got a little offtopic, but my main answer to your question was "if the company leaves the decision up to a majority of engineers saying 'yes', then a lot of companies do this." Google does this, the startups I've worked at do this, and some of my friends companies do this.
Reader beware, of course, I know at least one company on there shouldn't be.
I also found very few people could solve this (similar non-SF large city location). Occasionally, people who could not solve this were hired for other teams. Based on their performance, I don't think I would have been comfortable working with them.
I don't think it's unreasonable. But I'm not sure I'd use it as a screen if I was hiring now. I think I'd just have a chat and try and discuss a previous project. After that I'd move to a paid take home project (ideally representing real, useful work).
[1] Take a string, for example "ABCCABC" and count the number of times each 3 character substring occurs. In this case the answer would be 2xABC 1xBCC 1xCCA 1xCAB.
I froze up. I couldn't talk and think at the same time. I didn't have the skillset for doing this in a very intense scenario. In my case, I was homeless and needed a job ASAP. Every interview felt like life or death to me.
First one I was asked to reverse a string in C. I hadn't done C in a few months. I froze up on syntax. I looked like an idiot who couldn't do it.
I could imagine many people who have never experienced this format (or haven't experienced it much) would easily freak out and look stupid as bricks like I did.
I've since done over 200 technical interviews (as the interviewee) and I usually sweep. Still fail at FAANG but I always get the solutions. (Even the leetcode hard ones) Just not sure why I fail but cest la vie.
I realize people can still freeze up, but at some point I think there's just no solution to that unless the candidate can produce a significant portfolio. Also, I read your post and it sounds to me like to gained confidence in this area as your coding skills improved, which I don't think is as much a coincidence as you seem to believe.
Is there anything they could have done differently to make the situation easier? (I would guess perhaps giving you more notice about language requirements?). Or is there a different interview format that you feel would have worked better?
Stick to the basics and a trial period.
Perhaps some people might feel that while they could write code to solve this problem (or similar problems) outside of an interview. However, under the pressure of an interview they would not be able to solve the problem (due to anxiety, stress etc.).
If that was relatively common, then such a question would not be a useful interview question. Or would at least not be giving an accurate estimate of a candidates programming ability. This seems unlikely to me, but I would be interested in hearing different view.
That said, the last time I had a fizzbuzz take-home test, it was infuriating -- my code is online.. what does fizzbuzz prove except to help make a company soc2 compliant?
If you're dealing with user input or linting, then that's not a fizzbuzz test. I'm not doubting that maybe somebody asked you to do something larger than that including the fizzbuzz problem, I'm just saying that nobody else here is talking about take-home problems when we talk about fizzbuzz.
It's also my experience interviewing programmers (outside SF) that most candidates can't solve simple FizzBuzz style questions. As a take-home test, I agree I can't really see the value in it.
[1] https://blog.codinghorror.com/why-cant-programmers-program/
It’s astounding how many programming candidates can’t work their way through this.
also, someone who has only ever written code using lambda expressions might have difficulty writing a nested for loop
I just tried it, and it was trivial to do in a minute or two on my laptop. However, I did took note of two syntax mistakes that I made in the python REPL that were immediately obvious there and took seconds to fix, but which I most likely would not have noticed on a whiteboard.
So there's quite a bunch of problems where if the acceptable "format" of the answer is "hey, I'll just pull out my laptop from the backpack and push the solution to github in 15 minutes" then it'd be okay, but it'd be hard to do it while 'whiteboarding' without access to immediate feedback and easily accessible API documentation.
For example, I work in many languages, and for many APIs I can't remember whether in this particular language the same thing is called add or append or something else; e.g. I've worked on Java code for a dozen years and would be a quite productive Java developer, but since I haven't written any new code in Java for quite some time recently, I can't remember off of my head what's the right boilerplate to open a text file for reading in Java - it's something that a Java tutorial might have in the first pages right after Hello world, but I'd still have to look up the incantation to pass the encoding properly - there's like three Reader classes to instantiate and I don't recall their names off the top of my head.
It’s about your approach to problem solving and your ability to communicate that.
1. https://www.programmercoach.com/2017/04/interview-insider-co...
When I used this question I wasn't looking for accurate syntax. If it was a solution that looked like it would work, after some debugging etc. I'd consider that a pass.
Regardless most people couldn't answer it, which I considered surprising.
I've also had a couple FAANG onsites and get nervous as can be, but that's just a practice more and be more confident with my abilities problem I feel. No offers though yet. Still trying and practicing.
Id put out my fingers 3 characters wide to the first substring and step through the rest of it in 3 char 'spans' with my fingers. I'd say "extract each substring, put it into a map with that substring as the key and either 1 as the value if it wasn't already present, or if it was, increment the value"
I'd probably not bother mentioning how to extract the results unless asked.
If someone said that to me it would show the solution, and I'd be 100% happy.
I'd assume that they could then render that into code - perhaps that would be a mistaken assumption though but in my experience solving the core problem is the thing I'm interested in, not the syntax. But those with more experience may say the former doesn't always imply the latter, and the code needs to be shown.
What I struggle to understand is why this sort of thing still serves as a useful weed-out screen in our industry, even for people whose resumes indicate masters degrees and multiple years of experience in development organizations. But experience shows that there really are people who are ‘faking it’.
If so, then the company you're interviewing for must not be attractive to first-class CS grads (top 10-20% I think) from any non-online university.
As a comparison, any FAANG + Palantir/Jane Street/Two Sigma in the UK have tougher questions as their FIRST phone interview for INTERNSHIPS. (Palantir requires you to go through 6-8? interviews before getting an offer)
If this is a job that requires actual software engineers, I think rejecting everyone that failed this question would be perfectly reasonable.
If you’re saying only the top 10-20% of CS grads can answer such questions then that would make sense. I’d guess I was getting a random selection of candidates.
I'm saying that ALL top 10-20% of CS grads will be able to answer questions at this difficulty level in an interview scenario (accounting for nerves and so on). I'm not limiting it to them (i.e. the "only" quantifier).
I know people in the bottom of CS grads (judging by GPA) that worked remote jobs or worked on personal projects during uni that are very talented and would be able to solve this in less than five minutes. But there are also people that merely "got by".
> I’d guess I was getting a random selection of candidates.
I stand by my opinion that if you were recruiting for a software eng role (and not some random front-end position), then blacklisting anyone who failed this problem would be reasonable.
And, in my experience there are definitely situations where the majority of candidates can’t solve these problems.
One, these non-programmers are somehow not actually doing their jobs, but getting away with it. I've worked with people like that, but only a tiny number. Maybe i've been lucky.
Two, you don't have to be a programmer to be a developer. In this day and age, our tools, frameworks, and resources (ie Stack Overflow) are developed enough that you can actually get some useful things done without having to think mechnically. Knowing some obscure boilerplate (eg Spring annotations), how to use your IDE's autocomplete, and where to go for copy-and-pastable code for various kinds of problem actually makes you a useful team member! Even if you can't write a simple loop to save your life!
Do you get candidates that ever ask "can I assume the strings are ascii?" The usual solutions to this will break in amusing ways if you allow arbitrary unicode. I know some interviewers who actually would have as a hidden scoring criteria "candidate asked about input encoding", and a lack of asking about that is a fail even if they correctly solve the problem for ascii. I myself disagree with using such hidden criteria -- if I'm going to score something, and not tell the candidate what exactly I'm scoring, it's at least going to be something in the code like "correctly avoided the divide-by-zero case without me pointing it out" and not an expectation of the candidate to read my mind on my expectations of things I haven't told them. (I do tell them to try and write code without errors (I help fix up basic syntax quirks or if I spot a typo I'll point it out), or ask how confident in their code they are and whether they might have in mind any edge cases to try -- I'd like to get a candidate who actually writes a unit test on their own, I always point out junit is set up...)
You see my process better if you watch me write and correct this program on a computer. If my tasks is to think up a fully correct solution without iterative trying incorrect solutions, you're missing an important part of my process, even with a trivial FizzBuzz. (Though with this one you could probably just act as the interpreter and point out the errors for me.)
Can be done in O(n) time with a hash table.
- Treat recruiting in the same way as you do software development.
- Formulate a set of requirements.
- Define interview questions that give insight into whether or not the candidate meets those requirements. This is the equivalent of "tests" in the software process.
- Specific skills with your technology stack is good, but not necessarily essential.
- Ability to discuss sophisticated software concepts, and to explain software that they have built, and how they would build out ideas given to them is good.
- Evidence that this person gets stuff done is good (ref Joel Spolsky).
Coding tests are, for the most part, garbage. Not because the test is of no value, but because you the employer probably don't evaluate the result properly.
If someone could blag that while not being able to even write FizzBuzz, all I can say is well played to them.
IMHO this process works fairly well and does a good job of being economical with people's time.
No. Just programming is actually not easy and a lot of people apply for jobs they can't do.
Plus, a nonzero number of people freeze up in any interview situation.
I once failed an interview loop because I forgot how bucket sort works.
At the final interview to join the SRE team at Google I was asked to implement the kNN algorithm. I barfed at implementing a kD-tree after regurgitating the brute force solution.
Has any SRE ever had to implement a kD-tree in < 20 minutes or Google would go down?
I asked the interviewer at the end. They had never implemented one on the job.
As long as companies insist on these inane rituals I think it’s fair game to optimize for it as an interviewee.
It’s stupid but what else can you do?
- Gmail loads slower than Eudora did on dial-up
- Chrome takes so much memory it’s basically a meme now
- The Google homepage (a text bar on a white background) is several hundred KB.
So what I want to know is, if they hire so many algorithmic whiz kids, where the fuck are they hiding?
Chromium at least is open source. Make it use less memory.
a. hard workers, who will just do what is required without fussing
b. people who really want to work there, either because of the fantastic comp or because they drank the cool aid or they genuinely want to work on specific stuff that nobody else except the tech giants does.
I agree that it basically hazing, and has similar purposes - it discourages from applying people who are not hard workers or just don't care about working at a FAANG that much. As long as they still get tons of candidates, it's a great initial filter.
This is more true for startups that need less know "how to reverse a binary string" and more "how to properly design a database with 3rd order normalization", etc. If you're presenting an interview question, it should have actual job relevance and your co-workers should be able to solve it in the same time as the candidate. If you're drilling people on non-job qualities (e.g. invert a binary tree for a web dev role...) then you should expect a large difference between audience that can pass your bad interview tests and audience which will perform well at actual job.
Not trying to complain. I think job interviews should focus more on the 99% of what you do in your job on a Tuesday. Poor interviews seem to be more gotchas and algorithm tricks to disqualify roles which have little actual use for algos + data structures. The tests might seem too easy but as a front-end engineer I would rather be with a coworker which understands the CSS box model, knows semantic markup for accessibility, and similar web things than a person which is good at creating hash tables and doubly-linked lists in JS. Leet code probably doesn't test vertical centering techniques with CSS but if you're applying for a web dev position you better know them.
I felt this was much better in that it was less stressful, yet allowed me to demonstrate both knowledge and design skills.
Another company did something similar but more thorough: They invite candidates for a full day of work where they try to solve a small problem. Then the code is reviewed and evaluated together. They also start the day with a 1-hour overview of their current architecture and you get to ask questions and talk about alternatives. I think this gives both sides a better chance of finding the right fit.
I realize this is not always reasonable.
The first type requires the candidate to learn a lot about the existing code base. The second requires more mental work but is atypical of real work.
Implementing features sounds like a good idea. Perhaps create a small well defined system which is a simplification of the real world system and let the candidate implement some additional features.
It's possible that you asked the question poorly, or the solution wasn't as obvious as you thought.
Designing interview questions is hard[1]. I'll test out new questions on my peers at least two or three times before putting them in front of a candidate. And many don't make the cut. If a good engineer who's relaxed can't solve it easily, then a stressed out candidate will have no hope.
[1] This is why I hate seeing candidates share specific questions online. As an interviewer you'll have to scrub a good question, and switch to something you're not as familiar with. This hurts good candidates.
I think this was a great way to not only verify coding ability, but also testing team work and communication skills.
The art of behavioral questions isn’t “ask and answer” it’s the follow up questions. As you astutely point out, the questions are ‘stupid’. They might as well be “do you want a stick of gum?”
Next time you ask those questions do a couple of things. First keep in the forefront of your thoughts what information you’re trying to get out of it and keep the candidate on track answering your data point. Do that by relentlessly asking follow up questions. When you think you have everything ask more.
As an anecdote I was being shadowed during an on-site recently. I asked some arbitrary ‘dumb’ behavioral question, went back and forth a bit, wasn’t getting much out of it. I noticed my shadow clearly moving on to the next question in their notes and decided to keep pushing on the original question - why did you do this, what were you trying to solve, what motivated you. Turns out the candidate did all of this to generate new revenue for the company and ended up bringing in $10m a year extra at the small company there currently worked for. Loads of great data, would have never gotten there if I’d settled for the canned answer the candidate had.
Behavioral questions aren’t comp-sci trivial questions, you can’t just ask the behavioral equivalent to fizbuz/Fibonacci/floodfill and copy down the answer (and you should never be asking those questions either, but that’s a separate rant).
Behavioral questions are stupid and to some degree that’s the point. When you ask your significant other or kids “how was your day?” — guess what, that’s a stupid question too. What matters is what follows from your line of interviewing.
If you want to get good at behavioral questions listen to Fresh Air and try to be like Terry Gross.
> Is this kind of code problem too complicated in your opinion?
May you please post the problem so that we can provide you with meaningful feedback regarding said problem?
I agree that some coding problem needs to be used to try and answer the question, though with the right interviewer they can answer it without seeing code. The problems you use for that don't have to be at octree-collision-detection whatever challenge, a trivial nested for-loop is fine -- fizzbuzz level is fine. Sometimes you can rely on github or strong internal referral to skip this, but watch out, and anyway it's worth giving your questions to people you're sure will do fine (you've timed at least yourself right?) for the benchmark data and because sometimes they don't do fine, perhaps since maybe your question is too much. e.g. Floyd-Warshall can be done simply with a few nested loops, still I would never give it as a problem and I'd expect nearly everyone I've worked with to flunk it given only the standard hour (which really means 45 minutes).
Some jobs only need basic competence, so you might want to extend an offer if you've been convinced of its presence. At my last job, which ended up being more technically challenging / interesting than my current job, I was hired after posting my resume to Craigslist which led to exchanging some emails and having lunch with the startup founder to talk about my past work and whether I would be useful for his most pressing work. At my current job, I've been part of on-sites where I've established "can you even code?" is "no". Those were costly failures of not having that answered earlier. But we also like to believe we need more than basic competence, so rejections can still occur because of a lack of "testing mindset" or certain "behavioral answers". Only once you fix your "can you even code?" filter is it even worth considering what else you might want to justify an interview pipeline with more stages than a 'phone' screen or lunch conversation.
I used to wait to the first in-person interview to try simple fizzbuzz style questions (with the candidates on a machine and a compiler/interpreter). In about a third of cases that meant we'd committed a significant chunk of time to engineers that apparently couldn't solve trivial problems.
Now it's one of the first things I check. Done right, it's a relatively small hurdle for capable people to overcome, but really helps as a filter for those who aren't suited to the role.
I recently created a service (https://candidatecode.com) to help companies manage issuing and reviewing their coding challenges; I think it's got real potential to help some people out.
But if you need to recruit someone without any such credentials then you may need to do a simple coding aptitude test. Could be a code review or a simple excercise but whatever you do, don’t do whiteboard coding and don’t have people recite/implement memorized CS textbook algorithms. Anyone can do that and still no code.
If the person conducting the interview thinks the behavioral questions are stupid, then perhaps they are. In that case, don't ask "stupid" behavioral questions.
> Resumes are meaningless, and often re-written by recruiters to match the job anyway
Was the position entry level? Students coming right out of compsci often have little to no practicable experience. They may have difficulty thinking about what to put in their resume. After one or two years of full time experience that should no longer be an issue.
> For all I join in when complaining about irrelevant algorithmic questions, I have to admit that they at least test something, even if it's just willingness to study for the interview.
Asking those "stupid" behavioural questions and receiving the same canned answers also demonstrates a willingness to study for an interview.
The coding problem should be testing a candidate's problem solving capabilities as practicably required by the role being interviewed for. The chosen problem should reflect the types of problems that they will actually need to solve if hired. For example, you could select a small PR from one of projects being actively developed by the company. The selected PR should involve only one or two classes (assuming a language with classes) and require improvement. You can look through the history of a PR and just pull out a segment that was selected for improvement by the reviewer(s), or have the team select it for you. Then ask the candidate:
- to conduct a code review of the PR
- to improve the code
But we emphasized repeatedly we weren't looking for the O(n) solution, just the brute force naive solution was 100% OK. And it definitely didn't look like people were freezing up trying to figure out the optimal problem, they were struggling on the basic nested loop.
For instance, a front end engineer can be expected to be able to write a to do list or similar app in a framework (ideally the one your team uses, but not a hard no hire if not) app with minimal googling (although that’s fine as long as not excessive) in ~45 minutes.
Then you have to look at what level of experience they have. Less experience requires more mentoring generally, which may be fine depending on how much time your team budgets for that work.
Lastly, measure their body language and tone of voice to check for red flags pointing to difficult communication styles or people who treat others poorly.
If all three match, hire!
Any chance of a sample or maybe an alternate version of the problem?
I recently had one requiring me to develop a native mobile application, which I enjoyed. It was interesting, the code is useful down the line, and if I don't land the job, it beefs up my portfolio.
Initial screening by recruiters is tough as my background's missing a degree and industry experience.
Context: self-taught, started out with game dev, tried going solo - not a runaway success. Looking to move away from the field.
I find that this is very useful especially when interviewing juniors who don't have many projects under their belt for the first part. It's also useful when a candidate has good verbalisation skills, but poor programming ones (which happens).
Could you give the exact wording of your "nested for-loop" question?
If no one can answer the question, maybe they don't understand it? Maybe there's something unclear in the way it is worded?
Which I think is why a lot of startups locate in the expensive Bay Area - not a lot of cities have a similar concentration of decent talent.
You'd be surprised (or maybe not now) how many applicants for a senior frontend position can't build a progress bar for the phone screem.
I loath writing algorithm on whiteboard, especially the 'catchy' type. But I've interviewed people who can't even write a for loop.. the amount of brain drain in the flyover country is insane.
Could it be that you're having issues communicating the problem?
How did you run these? While it sounds like something that should be ok even on paper, you can vary comfort a lot through the medium. E.g. for the last interview I had with a substantial coding part, I think being able to do it on my personal machine made a big difference. (I obviously was told before what kind of environment I'd need to have ready)
I’ve been in a panel where I was the only person who asked a code question, the candidate flunked, and then the VP of Engineering went over my complaints and hired the guy anyways. He had been a Professor of Software Engineering and had a graduate degree from Princeton. Within three weeks, the VP of Engineering had to fire him because he couldn’t make it through a simple code review.
BUT the extremely negative sentiment here towards the technical interview process is very well-deserved.
Assessment of code (and the selection of problems) is most often no less subjective than any non-technical assessment. Sometimes the interviewer doing the grading is flat out wrong. Several times I’ve been asked the famous “given an array of stock prices, find the optimal buy and sell indices for the biggest profit.” One interviewer was not aware of the linear time solution to this problem, and didn’t believe me when I wrote and tried to explain it to him.
But sometimes, the interviewer doesn’t even want you to do well. Once time I was interviewing with an injury that prevented me from typing efficiently. I had a doctor’s note and the injury was quite conspicuous. Nevertheless, three start-ups made me solve problems by typing on a keyboard, which guaranteed an excessively long completion time. Those companies held those results against me. (An it’s not like there is anybody to hold those panels accountable).
And then there are those who just don’t care. I had a phone interview with Airbnb that was literally as bad as the stories on Glassdoor: the guy answered the phone in a noisy office (not a conference room), gave no introduction, then simply stated the problem and dumped me into Coderpad. I literally thought it was a prank, since I had met with people at Airbnb face-to-face prior to the call. But the recruiters confirmed the guy was a real employee.
The root problem here is there is no feedback loop back to interviewers. The candidates get “feedback,” but people asking code questions, especially new grads, typically get zero assessment on how well they are doing as interviewers. What’s worse is that recruiters and hiring managers both have incentives to deprive ICs of such feedback, since it would invariably make ICs more aware of opportunities outside the company.
Until the incentive structure of technical interviewing changes dramatically, we’re stuck with Leetcode and hope for the best. People like Gayle Laakmann are helpful (especially when Facebook gives candidates a live hour-long session with her for free), but these people are ultimately invested in their own income and not the task of fundamentally fixing this broken process.
dont give them a fizbuz. give them an example of a real problem your engineers need / are trying to solve. how do they respond? thoughts / intuitions / pseudo code. do they show knowledge of the problem space / domain?
ot if you just want a kid who can code and they pass the fizbuz but fail at the real job, what does your training/culture look like? who does that really reflect on?
it seems to me that interviewing is terribly cargo cult. the problem is real, the practises ostensibly supposed to be solutions are not.
/end rant
I was like this a couple years ago. I was a self taught, "college is a scam", "practical experience" type guy. I now, however do see immense value in the ability to be able to work through these algorithm questions,especially if you ever want to do something besides web / app development.
The classic detect a loop in a linked list question. The original guy took years to devise the algorithm for it. In an interview either you’ve seen it before, in which case you rattle it off, or you have to write a research paper effectively in 5 minutes.
“This problem is (analogous to) loop detection”
“This problem requires a sorted list”
“This problem can be solved by depth first traversal”
Actually sorting a list or doing a traversal is easy after that crucial step. You can look it up. You can’t look up the step that told you what algorithm or data structure to use though.
So am I smarter than Karl Popper? Not really. It's just easier to learn something than to find it. So, yes, I know the linked list answer.
What you want is very good football (soccer) players. Unfortunately you (or upper management) may not know all of the rules to soccer. You may not know the training regime that goes into winning a good soccer game - and it's a big risk spending money training for the big game on the off-season only to lose during the Big Show. So what do you do to test potential candidates? You see if you can get along with them, if they're a team player, and then see how well they play foosball.
It's perfect! There are soccer players on the field, there's a goal, it takes skill and coordination. But oh no, it turns out in the population at large really good soccer players really sort of suck at foosball. After all, they'd rather spend their time and energy playing soccer. So now they spend all their time reading up on books about foosball and what the best foosball strategies are.
You see where I'm going with this? No one uses algorithms in their day jobs. OK a few of you, but come on man, I make full stack web applications. As do most programmers. So why are you testing theory that has literally nothing to do with the job? Somehow someone thought this was a proxy for smart people, but I mean, if the guy who wrote this "Tech Interview Handbook" was really smart wouldn't he have spent his time writing a cool program? I mean how lame is this?
If you want to hire competent engineers tell them what you're building and ask them how they feel is best for them to show they're competent. If you have a big data pipeline in Scala ask if they can construct a data pipeline example that is cool over a few days (take home) or do something similar in the office. Some people like one, some like the other. But just communicate with the people you want to hire! And if not everyone's interview process is the same then maybe that's ok.
I mean I just got a guy who sent me some automated code interview program that had a timer that counted down from an hour at the top! WHO THINKS TREATING THEIR POTENTIAL EMPLOYEES LIKE STAR WARS DRONES IS A GOOD THING? All you have to do is treat the people you want to work with like you would want to be treated and demand that they know their shit.
This. Isn't. That. Hard.
I just had to conduct a round of interviews in a non-SF large US city, and it was a hellish crapshoot. Resumes are meaningless, and often re-written by recruiters to match the job anyway. Everyone has the same canned answers to the stupid behavioral questions. And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it.
Is this kind of code problem too complicated in your opinion? For all I join in when complaining about irrelevant algorithmic questions, I have to admit that they at least test something, even if it's just willingness to study for the interview.
Instead of reading everybody's complaints about interviewing, I'd love to hear how you think it should be done. Because I have to admit I'm pretty much lost right now.