NOBODY HAS "FIGURED OUT" HIRING !

Not Google, not Apple, no one. Sure, some places (and individual interviewers) are better at it than others. But at the end of the day, hiring is a deeply subjective process with lots of error and uncertainty built into it's nature. The subjectivity is intrinsic.

Places like Google can afford to be nonsensically picky and not suffer drastic consequences from it. They have a thick, never-ending stream of highly qualified candidates. At their volume of hiring, it doesn't matter to them if they screen out some folks that would have been brilliant hires, nor does it matter if they hire some promising but ultimately disappointing duds. All of that is OK.

Sadly, however, it seems that small shops are trying to cargo-cult Google's hiring practices. That IS harmful to the company and the candidates, IMHO. I think folks in these non-FAANG companies should get trained on how to conduct interviews, especially if they're interviewing non-senior candidates. Interviewing is a skill in itself. It's not something that comes automatically with expertise nor is it something that can be left entirely to HR drones.

Let's say Google would like candidates with PQ>130 with 95% confidence. Google has an error with std. div. of 15 points in measurement of PQ in jobs interviews. Google then needs to set the hiring bar at 160 PQ in order to get those candidates. This:

- screens most qualified candidates out; but

- most candidates who do screen in are qualified

Statistics would suggest this leaves you with 95% qualified candidates. A more precise Bayesian analysis will show you don't end up with 95% qualified employees, but the basic idea works -- it's still a majority. You set an impossibly high bar, so that candidates hired need to be qualified AND lucky. You discount unlucky candidates, but you don't hire (many) unqualified ones.

The problem, of course, is that all Googlers are convinced they all have a PQ>160, and are superior to everyone else. That's where you get the obnoxious Google incompetent arrogance.

Someone just released a product that offloads Chrome to the cloud. Gmail has a very long loading screen. Hangouts was replaced by something like 4 incompatible apps. Android phones are significantly less power efficient than iPhones. YouTube copyright notices are trivial to game. Etc

Most of what makes big companies succeed or fail is in the overall culture, organizational design, incentive structure, and corporate structure -- properties of a network of individuals rather than of those individuals themselves. I think most of Google's success and failings can be explained that way, much more so than the success or fault of employee quality.

Organizational design is really hard to get right. A senior manager described it like a herd of cats. If you get them all mostly moving in a beneficial direction, you're doing okay.

That's why they pay executives the big bucks. Executives fake understanding how to manage this stuff. Most don't, but they do a good job of convincing boards that they do.

I wouldn't care about some company's code quality but in these cases Google's clout (due partly from their maladaptive hiring practices) causes these bad libraries to get grandfathered into many projects that I have to deal with.

I've worked in companies where great programmers produced horrible code, due to cultures of optimizing to productivity metrics / features shipped, rushed timelines, and interrupted work schedules with meetings, requirements changes, and people multitasking projects.

I'm not arguing one of those is better than the other. Running a business is about tradeoffs. I am not particularly impressed with anything Google has engineered in the past decade or more. The original search, gmail, Google Docs, Android, Maps, and a few others were brilliant, but those are a long time passing.

On the other hand, I'm not ready to condemn anyone working on those over that. Competence is situational and context-dependent. I also don't have insight into Google business decisions. Google revenues are growing exponentially, so they're clearly doing something right.

Makes me curious about if you have thoughts about how to find people who are good at that stuff for real (not just faking)

And likewise, other employers looking to hire ex-Googlers are convinced of the same.

Place the bar high enough and you will get more and more people far into the tail, and less and less people that actually fit your bar.

Corporate arrogance isn't a property of individual personalities. Most Googlers are perfectly nice people. The Google corporate culture is a whole is rooted in a deep superiority complex and dripping with arrogance. Google believes it knows better than its users, and that translates to all aspects of product design. If you moved those same engineers to a different company, you wouldn't have the same behavior.

I'll also mention that each organizational design has upsides and downsides.

This culture seems to work well in Google's early markets (e.g. search) where users are statistics, and where most problems are hard algorithmic problems, and users are secondary. It has upsides in B2C markets like Google Docs or Android. It crashes-and-burns in a lot of B2B markets, like Workspace or GCP, where customers have a high degree of expertise which ought to be respected.

I'll mention a lot of fintech companies, as well as elite universities, have a similar culture. Those are domains where it leads to success as well.

Instead, you should try to find lots of correlates of PQ and measure them regularly.

A lot of really good people are discouraged by repeated rejection. However, high levels of rejection of very qualified people are built into this (and many similar) systems. You have to be qualified AND get a lucky die roll to get in the front door. Once people stop taking rejection personally, they can start acting more rationally, and there's less emotional harm. People feel really bad about themselves otherwise.

There are back doors with less luck involved.

Because of this, I often hire people I've already worked with. I'm also often hired by people who've already worked with me. I hate that this leads to a very homogeneous experience... or even what might seem like gatekeeping. I've just found that the best indicator of how a person will perform, is already being familiar with their work.

The good news is, it's a very open network. We have a highly active Meetup scene, pretty regular public hackathons, annual small software conferences and un-conferences, and coworking spaces are (well were) packed.

For the most part this has worked, people new to the community are able to find jobs and the people hiring them know what hey are getting. But also like you say, this doesn't really scale to larger operations.

I'll accept the statement. But I will say that hiring from a network is at least a very different thing from hiring through a grueling set of often artificial interview hoops. It requires a potential candidate to have genuinely interacted at a higher than superficial level with a lot of people in a professional capacity. Which may not be harder than "leet code" but is certainly very different.

And yes, all the big companies have referral programs but that's mostly just a very rough first pass as a lot of referrals are basically I'm connected with this candidate on linked in. Referral bonus please.

In my, admittedly not super extensive experience, I tend to believe that about 1 in 10 companies actually knows how to run a hiring process. I'm not sure precisely what kind of error bounds I'd put on that, but I doubt I'm off by more than a factor of 2 either way.

This might actually be part of the retention strategy. If everyone who works at FAANG knows that they're somewhat lucky to get in, regardless of ability, it means they are less likely to jump ship. What's the point in applying for another job when you're already paid well and it's unlikely to end in anything but lost time? You're unlikely to win a 10% lottery twice, given the amount of time you'd bother investing.

I am not looking externally until I'm sure I'm done at AWS. I've done 80+ interviews here, and I'm consistently amazed by what drives people to say "The candidate wasn't up to the coding bar". As far as I can tell, you are almost never up to the coding bar. I think we interview so many people just to remind ourselves that if we leave, we aren't getting back in.

Just to head off the "So why don't I argue for change?" questions, I'd rather work on self improvement than fight tooth and nail to make the hiring process a little bit better. I'll let somebody else add that to their promo doc.

I never interviewed for the AWS side of the house since I knew I wanted free time, and a life.

It's the bar raiser's job to make sure that the hiring manager doesn't just hire people to fill seats. More than once, I've seen bar raisers override hiring managers when there was compelling evidence from the technical interviewers.

At least in my org, you don't get the job if you don't raise the bar. Period.

I had plenty of times where an entire hiring pod would incline on a candidate because they had the soft skills we were looking for in that role and we knew we could sharpen their technical skillset in house. You don't have to be a rocket scientist to work at a FAANG... you just have to be interviewed by a team that is hungry and likes you.

You're evaluated against people in the same role, not the entire team. But, yeah, the idea is that the hiring standard (the 'bar' in Amazon terms) is always getting higher.

Same as there is no amount of money you could give me to end my life, a position which I assume is shared by a vast majority of humans.

Time is not money. It’s not even close as an exchange rate. I would never put myself through these types of interview processes (not to mention that I would assume that sort of thing to be indicative of the job itself and company as a whole) because I value my actual life and dignity far and above what I value as an upper middle class income.

I am at least 90% sure that part of these interviews is to filter out people who these sorts of views. These employer would much rather have someone who wants to work 100 hour weeks and be the 'hero' over someone who works exactly 8-4 monday-friday and then goes home.

Pretty sure a lot of older parents would take a deal with lethal injection + $1M for their kids to inherit.

That always works out well.

The 'FA.N.' part of it maybe: https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...

If I change jobs, and find myself in a good situation, and need more head-count, I'm going to reach out to people I like that I've worked with before, see if they might be interested in a job change. I know I can work with them, know we'll build good stuff.

There's a constant drive in FAANG to hire more people, most teams have open headcounts. To the degree that it's extremely hard to build up that funnel of candidates. If my team has a head-count of 5, that realistically means I've got to find a bare minimum of 30-40 candidates to enter in to the pipeline from somewhere, to maybe get close to that target. That's a slightly optimistic conversion rate. Now scale that up across a company with thousands of teams that are hiring. Getting that pipeline filled on that scale is crazy. It's just one of many reasons why FAANG go all in on college hiring events.

I can almost 100% guarantee to get someone I've worked with in a FAANG co-worker through the interview pipeline and in to a job. They know their shit, they know what they're doing, and they know what the process is like.

[1] https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...

Part of the problem is the sheer scale of hiring, but I think most of the problem comes down to the lack of feedback or evaluation mechanisms on the interviewing side.

They train you up, half a days worth, training tells you not to be an arsehole, not to ask stupid questions, not to have unreasonable demands of candidates, not to be biased. They don't train you in valuable skills like active listening.

Next thing you know, you've done training, and you're interviewing candidates every week or two (or more often), and there is zero feedback mechanism. No one evaluates your interview questions, no one asks candidates to provide feedback on the interviewers. No one looks to see if you've got unreasonable expectations as an interviewer, or have your expectations set too low.

You do have to do a post-interview group discussion with the other interviewers to make a yay/nay decision, but it's super easy to present what you did/asked in a positive light.

The whole system is designed around the interviewer being right and infallible. Is it any wonder the process is so completely and utterly broken?

edit: > or have your expectations set too low

This is where I bias towards in worrying, imposter syndrome and all that jazz. I've made a conscious choice to not raise that bar higher. I think the questions I ask are good, I think they're set up well enough to encourage candidates to go as deep as they feel comfortable with. I try to design them with no one true answer but have a few in mind so I can go where the candidate goes.

1. We trained 4-5 times on each type of question. The first few were shadows, and then we did reverse shadows where someone watched us give the interview and gave feedback later. In one category I asked for and was allowed to reverse shadow an extra 1-2 interviews.

2. There was auditing. In debriefs where you discussed the candidate and reviewed notes, the debrief lead was supposed to closely examine what questions you asked and how you conducted the interview, with the explicit goal of making sure that the interview was conducted within spec and your recommendation made sense given performance. Shortly after I was certified to do interviews, a debrief leader (correctly) identified a major issue in an interview that I had conducted. That candidate was given another interview in the same category. Although I didn't face any official sanctions, it was definitely an embarrassing experience and made me handle future interviews more thoughtfully.

Overall, I was fairly comfortable with the rigor of the process that I saw. I'm certainly not saying the process is perfect but my experience did not align with yours.

(the "lizard brain" is a myth by the way; I think you're confusing it with unconscious bias of heuristics, which is not what the triune brain theory was about)

Are you saying each interviewer just makes up their own questions?! That's ludicrous if so. Where I work, we have a standardized pool, with standardized evaluation criteria.

Obviously the drawback for FAANG is that standardised questions would rather rapidly leak. Very quickly you'll just end up with candidates that know how to answer your questions.

Where I work now, it's a mix of pool questions ("soft" skills) and interviewer-made questions (technical skills), but it's not a hard and fast rule to use the pool questions. I rarely use the precise wording for the pool questions, and instead adapt them to match the conversation with the candidate.

Whatever metrics that google is using in their interviews have probably become worthless in the past decade as people game the system.

The candidate hated the interview, claiming it was discouraging. Coding is erratic, talent is strange, it really is a craft and we still don’t know how to reliably raise someone to competency.

1. The candidate was a complete and utter fraud and their previous (and apparently well-regarded) employers were too stupid or negligent to notice this, wasting literally millions of dollars (48-21 * $100,000+).

2. Something about the interview failed to let this person demonstrate the skills that had kept them employed for two decades. Maybe their mind went blank under pressure, or at the end of a long day. Maybe they got hung up on something trivial (that a quick search—-or nudge from the interviewer—-would have resolved), or the question was unclear.

Fizz buzz was a great question in that it had pretty much a Boolean success criteria.

If the interviewer only says "Write String.contains that passes these test cases" goes back to playing with their phone, several things may happen. One person will take that absolutely literally ("It's a test; better do as I'm told"), and you'll dismiss that apparent garbage or move onto the "real" assessment where they're hoping to shine.

Another will get bogged down in something the interviewer regards as a distraction, and "waste" a bunch of time on something the interviewer regards as a distraction. "He handled unicode, but not substring matching (KMP or Booyer-Moore) or vice versa." Maybe someone will goldilocks it and hit the right (not-explicitly-specified) balance of (also unspecified) features and time, but...

If you structure it as "Please, do the dumbest possible thing and we'll iterate"--and don't hold that initial pass against them--I could see it working well.

That would tell you if their workflow would fit your company much more than knowing how to run a coding challenge would.

[1] https://wiki.c2.com/?FizzBuzzTest

There are certainly a few. The job market, being a combination of people who want new jobs and those that can’t keep their old ones, is undoubtedly enriched for them.

Even so, it seems unlikely to me that there are anywhere near as many as most people say. You certainly don’t have to hire someone who flubs your interview, but you also don’t have to assume they are frauds.

30%, minimum. I was hired alongside a guy with a fantastic resume. He pushed zero lines of usable code in 4 months. When he left I purged about 20 files which were tests that were just completely commented out (but I guess those count for LOC according to github's crude measure). I would say that not only was he incompetent, he was worth negative (thankfully, nothing critical) - Maybe in order to cover his tracks? he had moved certain classes of tagged tests (e.g. skip, broken) to "ignore" status instead of 'yellow star'/red dot, I now, months after his departure, have a pr reverting those changes months after because I didn't notice he had done that. Thankfully it had not covered up any major defect in our codebase (someone could have left a corner case test as "broken" with the intent to fix it later and wound up forgetting to and sending it to prod).

But hey. Programming isn't that bad. In the physical sciences it was 60-70%.

How was this not uncovered during code reviews?

I think the reason why every developer tends to have a story about these sorts of incompetent colleagues is not necessarily because 50% of their colleages are incompetent, but because even if just 2% (one person in the department) or 5% (one person in your larger project team) is incompetent, that can be enough to cause a seriously negative impact.

I’ll estimate zero to 10% wildly incompetent. Many of the folks who aren’t able to program find other ways to be useful: Testing, requirements, prod support, sys admin, config. It’s not even clear they couldn’t program, but maybe came to prefer the other work at some point.

What’s your wildly incompetent estimate?

> The job market, …, is undoubtedly enriched for them.

It is still a very useful test.

Personally I am a much bigger fan of using FizzBuzz as a gate than an algorithm question. I think algorithm questions optimize for the kind of developer who doesn't mind memorizing algorithms to get a job, which might be a useful skill, but you can test that same skill of memorization using FizzBuzz, and then you don't end up also filtering out people who can code but don't care about memorizing algorithms.

In any case, I always think it's worth using their solution as a jumping-off point to ask other, more language-specific questions. Things like: how would you change this if it was intended for use in a FizzBuzz library, how would you annotate this if you needed it to be injected as a Spring dependency, why did you use a for loop instead of a Java 8 stream (or vice versa), what are the implications of declaring this thing as final or static, can you write a unit test for this, and so on. That's when you can get past the point of memorization into figuring out if they actually understand what they typed, which is helpful to ascertain their level.

well, I mean, you get to ask what you like... but this is how you determine if someone understands what they've typed on a conceptual level?

For me there are basically two questions to answer when I am interviewing someone. The first is if they have any real programming ability at all, which hopefully FizzBuzz should answer. (Many people do not pass that threshold.) After that I'm looking to figure out where they could fit into the team, or the company. That means seeing if they are already familiar with the frameworks they will be working with in the position (usually, but not always the case for junior applicants who have held at least one job before), but then also if they can speak critically about some of concepts used in those frameworks, and perhaps compare different approaches that have been taken to solving similar problems over the years (if they are more senior).

It's not a wrong answer if they don't know the framework or the concepts behind it at all, since they might be switching specializations, but that's important to know at the interview stage because they might be better suited for a different role than someone who is deep in the framework and more likely to be able to hit the ground running.

My question is: what happens after people pass fizz buzz? Failing fizz buzz is how you filter people out, but it's unlikely that coding up fizz buzz passes the technical screen. What kind of questions do you use to establish this, once you're past fizz buzz?

I've failed far more tech screenings than I've passed. I could easily do fizz buzz, and when I've prepped for an interview, I could some tree and set permutation stuff. But the questions get so much more difficult than this. Since difficulty varies, an example of a difficult question for me is "find all matching subtrees in a binary tree" (at the whiteboard, in 45 minutes). When I got feedback about the no-hire, the explanation was that I had a good grasp of algorithms and made some progress, I didn't solve enough of the problem in code (tight pseudocode would have been ok) in time allotted (again, this was ~45 min at the whiteboard, one in a series of 5 one-hour technical exam style interviews during a day of interviewing).

I can't claim to be a great coder. I have understood how to code merge sort and quick sort and more complicated tree structures, and I could do them again if I studied and loaded it all back into short term working memory, but I'm content to know how the algorithms work generally and get back into the details when I need... but when anyone mentioned "Fizz buzz", I do insist on stating that my impression, based on quite a few interviews, is that fizz buzz isn't what is screening out software engineers. Lots and lots of people who can write fizz buzz (and build and print a binary tree pre order and post order, and do dfs and bfs, and solve problems with them) are still frequently screened out.

I'm at the point where I just won't do tech interviews anymore (or take home tests). I won't study for exams or do mini capstone projects for an interview that may or may not work out. I would do these things for a degree or licensing exam, but not for a job interviews. It's just too much of a time sink.

I accept that this may cost me good opportunities (in fact, it has), though of course I don't know if the interview would have gone anywhere, other than costing me another long prep session with "cracking the coding interview".

I'll finish the way I usually do, by 1) acknowledging that you are free to interview how you like, and that nobody owes me a job, and 2) mentioning that many companies complain incessantly about hiring difficulties without realizing that their own interview processes may be filtering out talented people and that nobody owes them an employee either.

We tune up a little bit the difficulty. The point is to start a conversation and see what the candidate knows about, ¿does he knows about time complexity?¿differences between passing by reference/value?. Afterwards we talk about the technology that they use, what they like, and what they will like to use in the future. Just to see if they read about their field and are able to talk without saying something egregious.

And if they do fine we bit the bullet and hire them.

> nobody owes them an employee either

Now that I am "in the other side", I can see a lot of things that will definitively would improve the problem at micro/team level, like posting salary or increasing the WFH days. But ¿would they improve at macro/company level? The things is, companies, including tech companies, from small startups to big corps, usually have much more problems than the quality or quantity of their software.

(signed, someone that has been rejected, and will be rejected, to more interviews that he has passed)

This was the introductory question before launching 200 threads and asking him to solve the deadlocks/inefficiencies, which was the real question supposed to let him show off his skills in front of my employees, specifically crafted for him because I wanted to persuade my employees he was an excellent hire. So he had a taylored chance to show off his skills but failed at the introductory question.

But on the other hand, how can you be asked “Here’s a substring, return true or false if it contains the substring, this is the introduction of 5 questions so don’t sweat it” and not just write two nested loops and an if? I’d pass on UTF8 problems, but when you’ve been working with Java for CRUD apps, you still should have your UTF8 correct. This is how you end up with passwords that must be ASCII because the programmer is bad.

People's brains just occasionally lock up.

I'm sure I'm on somone's list of incompetent bozos for an interview that went like this: "Please, describe Python programming language." That's it. I had no idea what I was supposed to be doing, and the two fellows interviewing me would not elaborate.

I talked about what I had done with Python. Stony stares. Do I talked about the nature of Python itself (interpreted, multi-paradigm, lexical/LEGB scope, the GIL). Stony stares. I wrote some trivial programs on the board. Stony stares. Had I brought an actual snake, I might have tried to charm it.

At the end, the CEO told me they weren't overwhelmingly sold on me, but would think about it. Never heard from them again.

There are so many "developers" just faking it, I can certainly understand using a test that would reject 90% of the good candidates if it could reject 99% of the bad ones.

And if the candidate was able to do this in 6 minutes, what would you have thought? "Great, let's hire"?

In my humble opinion, the question is a waste of time either way. You'll get much further trying to probe what the candidate does know rather than randomly creating an exercise that you think they "should be able to do if woken up in the middle of the night". People forget how stressful interviews are and how easy it is to assume shared context.

The fact that your employee was able to do the test might be indicative of the fact that you share context with the employee that you did not share with the candidate, thus confirming your bias.

Multi-lingual support seems really really hard, especially in six minutes. I would think most people would need to look at technical (i.e., unicode) and linguistic references to get it right.

Should does the ligature f l match itself, or the ASCII constituents 'f' and 'l'? How about combining vs. pre-composed characters? Some Chinese characters show up in other languages (Japanese, Korean) and are sometimes split between Hong Kong/Taiwan/Mainland language tags too. In fact, there's a mess of work devoted to this ("Unihan" https://www.unicode.org/versions/Unicode13.0.0/ch18.pdf). Having figured out what you can do, you then need to decide what you ought to do. Not being a Chinese-speaker, I have no idea which options would seem natural....

In fact, having written this all out, there's no way someone "solved" it from scratch in six minutes. It would be a great discussion question though....

    for (int i = 0 to text.length() - substring.length()) {
      boolean found = true;
      for (int j = 0 to substring.length()) {
        if (text.charAt(i) != substring.charAt(j)) {
          found = false; break;
  }
      if (found) return true;
    }

> What was the goal of the question?

This is the introductory question before solving concurrency problems, because it’s much easier to understand what a thread does when you’ve coded the body yourself.

> Why did you want the person to implement a contains method?

The job is CRUD + integrating with Confluence + parsing search queries from the user, so finding “<XML” in a page and answering “Yes! This is totally xml, I’m positive!” is a gross simplification of realistic tasks in the real job (and in fact in most webapps), with characters instead of XML or JSON.

I have the feeling that you think this question is entirely abstract, but I both tailored the exercise because he touted being good at improving app performance on his resume (including using JProfiler) and I took care of using a realistic on-the-job example.

> Did you really want to verify they understood String implementation in Java?

Well, what consumer product can you work on if you trip into all UTF-8 traps? Telling customers “Just write English because we can’t be bothered to learn the easy thing in Java that handles UTF-8 properly” is… is acceptable unless he also fails the fuzzbizz test. And once UTF8 is mastered, it’s good for life! I wouldn’t mind teaching him if he didn’t fail the rest, but as a senior you should really know the difference between .getBytes() and .codePointAt(i).

> If the candidate was able to do it in 6 minutes, what would you have thought? “Great, let’s hire”?

The 4 other questions were classic gross concurrency errors, tailored because he touted it in his resume and I wanted him to shine. A senior should be able to guess them blindfolded as soon as I tell them “There are concurrency problems”, without even looking at the code ;) Volatile, atomic, ArrayList non-synchronized, 200 threads for a connection pool of 10, a DB accepting 7 cnx (note the prime numbers make it easy to spot which multiple is causing the issue), and strings of 10MB each with Xmx=100m, if he finds any 3 of the 12 problems, and 2 more with help, I’d hire him. If he ditched the code and postes tasks into an ExecutorService (as they teach in the Java certification level 1), I’d hire immediately.

1) We want to test concurrency but start with implementing String#contains.

2) You have to know how to implement String#contains because you might use contains in our environment (not really, but theoretically, so you better know how to implement it).

3) You must absolutely avoid basic UTF-8 traps because users use UTF-8.

Neither of the above tells me what would you gain if the candidate nailed the question. It just tells me that:

- Your team might or might not use contains to verify something is XML (I truly hope not).

- Your team uses UTF-8 strings (which is one piece of the shared context that the candidate probably does not have).

- You tested candidate abilities of performing under pressure rather than testing their knowledge or skill.

- You are trying to hire the exactly same senior developer as if you promoted someone on your team with your codebase.

You come to the interview full with assumptions and biases about what a senior candidate absolutely must know instead of seeking what they bring to the table and why they call themselves senior. Let me tell you there are lvl 4 and 5 Java candidates that have never touched UTF-8.

Finally, and let me blow your mind here, there are senior developers that haven't really used String#contains in the last X years of their career either.

I don't know what was the quality of the candidate, but I feel, from my limited PoV and lacking all the info, that your interview process is deeply flawed.

I also completely fail to see what a CRUD app (i.e. java + db) + shooting REST requests to confluence has to do with your concurrency questions, as in interview != job fit, but that might have to do with some missing context.

This! When giving interviews last I really worried if the questions I asked where just indicative of my own Dunning-Kruger effect.

i.e. Do I only ask questions I already know the answer to and not questions I don't know the answer to?

If I do then am I just filtering for people with the same background and knowledge and missing out on people with other skills I don't know, because they're in my blind spot, I need yet?

I've been advocating for a "coding interview" where both the interviewer _and_ interviewee draw some random question from leetcode or other problem bank, and try to work at it together.

This would show collaboration skills, and you can tell pretty easily how helpful the candidate is with his/her contributions, and whether you find there is an impedance mismatch somewhere.

It probably also maps more closely to the kinds of interactions you'd have after the person's been hired.

I think it would also help calibrate: if you can't figure it out, is it fair to expect the candidate to figure it out? Maybe it's just a hard problem!

My then-co-workers liked to ask harder algorithmic questions but I wanted to give the candidates a little bit of a break with something easier.

It didn't always work, but at least I tried.

What were you trying to tease out from the problem statement?

I am not a Java programmer but it took me 30 seconds to find the contains() method.

And the NDK clearly is anything but modern C or C++.

I’ve heard it referred to as “C+-“.

Apple mostly cares about LLVM based tooling in the context of Objective-C and Swift, Metal is C++14 dialect, and IO/Driver Kit require only a subset similar in goals to Embedded C++, so that leaves the remaing of the clang community to actually provide the efforts for ISO C++20 compliancy.

https://en.cppreference.com/w/cpp/compiler_support/20

Which one is best for open source is debatable.

Permissive licenses make it easier for companies to make proprietary software, or even a closed source version of the original project.

Copyleft licenses (like GPL) are intended to promote free/open source software but they can be a legal headache, which can make users favor proprietary solutions.

On HN, I think that people tend to prefer permissive licenses (but complain when large companies "steal" their work, go figure...).

"Data-oriented programming" (to distinguish from object-oriented) is largely C-style C++ that is written for performance rather than reusablility/abstractness/whatever. In the embedded programming world where performance is paramount, a lot of people have low opinions of many C++ features. One could also never completely trust compilers to implement everything correctly.

I come from an embedded background, and understand that.

Aren't they the company that pretty much requires an Ivy-League sheepskin to be a janitor?

"The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt. – Rob Pike 1"

"It must be familiar, roughly C-like. Programmers working at Google are early in their careers and are most familiar with procedural languages, particularly from the C family. The need to get programmers productive quickly in a new language means that the language cannot be too radical. – Rob Pike 2"

Sources:

https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Pa...

https://talks.golang.org/2012/splash.article

My pet theory is that HR minions feel compelled to portray their role in the process as something that adds a lot of value and outputs candidates which meet a high hiring bar, even though in practice they just repeat meaningless rituals which only have a cursory relationship with aptitude and the engineering dept's needs.

This is the first time I've been involved in coming up with the process, but from what I've observed, it's a similar situation in other organizations.

That is simply not how things are. The hiring bar is designed and upheld by people on the same software engineering job ladder as a candidate. The role of recruiters is primarily coordination. The role of HR is compliance with local employment laws.

Makes a lot of sense, you could solve all these questions without knowing specific algorithms as long as you are good at problem solving - which is, I assume, the intent of the process.

Obviously it doesn't always work like that.

You could solve all these questions as long as you are good at problem solving, *given enough time*.

However, with tight time constraints and perfomance pressure, the only way you could solve all these questions is memorizing and practicing all these algorithms.

Apparently the correct answer is to use "Robert W. Floyd's tortoise and hare algorithm", which is trivial to explain and code.

The catch?

It took a decade of computer science research between the original statement of the problem and Floyd discovering the solution.

So... no worries, you have an hour, a marker, and a whiteboard. Your time starts: now.

"So we specifically asked for linear time."

"Uh, yeah, I did it in log(n). That's better."

"It doesn't match what's on this paper they gave me. Thanks for your time. We'll be in touch."

You’re expecting the mechanical engineer to recall something they learned about kinematics, not derive it on the spot. It’s a test of knowledge, rather than cleverness. The equations of motion are also more central to physics and engineering.

A decent programmer should know that linked lists exist, their general properties, pros and cons, etc. However, cycle detection is not a particularly common operation, so not knowing Floyd’s algorithm tells you very little—-and their failure to do years of research in 45 minutes even less.

The tortoise and hare algorithm is not the foundational skill required to make software work the way an understanding of motion is for building structures. That's why it's often omitted from educational material yet these people are able to produce usable software after even something like a bootcamp (which I guarantee basically no bootcamps ever touched this algorithm).

I'm not sure I approve of asking even more well known algorithms like Djikstra's algorithm or A* in a job interview, unless the role was something that specifically required that area of knowledge like building pathfinders for video games or robots or something.

Also, it took around 13.8 billion years for Newton to do what he did.

If they actually aren't looking for people who just cram CTCI or leetcode, coming to this answer from first principles is demonstrably far more difficult than you'd expect achieved in an interview.

These sort of questions have an incredible recency bias, and have zero relevance to engineering competence.

Of course; how else do you do back-door age discrimination?

No such avail. In fact, unless these algorithms and problem solving methodologies are baked into your memory there's no way you are white boarding a Leetcode hard level problem in an interview.

What I was impressed at an Uber interview was their system design interview process - which basically boiled down to 'how do I abstract retrying a 429 - rate limit exceeded.

What I take is that - the interviewer is expecting a very specific solution even in an open ended system design question. It's like throwing a needle in a haystack at you and expect you to get to the needle in like an hour :).

They're probably looking for some sort of variable refilling leaky bucket implementation, which is funny because I believe this is exactly what they do internally. It was probably the task the interview had in front of them in their day-to-day and wanted you to do it for them!

This is a fair design question for a senior role (which this sounds like) that promotes disccussion, but expecting a specific solution is really only testing "does this person match my preconceived ideal for what a <dev> is?" which is really dangerous and has very little value.

I thought it was exactly those who Google wants to pass. Anecdote: ex-colleague of mine who is not specially bright studied 3 months how to "crack the coding interview" and got a job at Google. His knowledge about algorithms and data structures was like mine: I know what a tree is, I know there exists operations one can perform on them and some of them are more performant/efficient than others... but I would need to Google how to "reverse a binary tree" if I had to do it in less than 1h.

The biggest difficulty for most seems to be that they don't know what "reverse a binary tree" actually means. It sounds kind of mathy and opaque, so I get it, but candidates should be able to have a dialog to figure out what the requirements mean. And on the flipside interviewers should be ready to have that dialog and not count not knowing the term by heart against the candidate.

This problem to me feels qualitatively different than the "rabbit and hare" algorithm for finding a loop in a linked list mentioned by another poster. That one needs a non-trivial algorithmic insight that just might not come to you during an interview. The solution to "reverse a binary tree" flows out of the structure of the problem statement as long as you have the fundamental skills for walking and manipulating data structures and the conversational skills to understand the problem, both of which seem fair to test for.

Sounds bright to me.

I think your weights are way off.

Plenty of times where I've used trees because they're the logical representation of the problem (ever had a field called "children" in your code? HN comments are a tree. Etc.)

While it's probably not what an interviewer is looking for, having the most common solutions memorised gives you an advantage of time. A coding interview usually consists of two challenges. If you get stuck on the first one and take too much time to answer it, you won't have enough time to go through the second one.

To avoid the code printer perception you can always go through an explanation what alternative solutions could be applied to the given problem, what their complexities would be and why the one presented is the best.

And yet, they routinely do pass these interviews.

That's just absurd. If someone with no "algorithm" experience who was a good problem solver had to work out an answer from scratch for the interview its almost certain they're going to find the brute force answer and FAANGs pretty universally want the most efficient one so these people would routinely fail.

Its totally clear that what FAANG hiring optimizes for is recent CS graduates who passed tough Algorithm weed out courses at well known colleges in the past 18-24 months. They are young with no families or obligations and are happy to work 12 hour days at Google because they have hip open offices and ping pong tables.

No one knew who they were interviewing or what was on the resume until they glanced at it while walking to the interview room. I suspect all interviews and the process are just made up as folks go along, and half the time you get a gig or not mostly based on if one of the people you talk to is in a good mood or not.

This is in contrast to tailoring each interview to a candidate's background.

With that, I think looking at a candidate's resume 15 minutes before an interview is not that unreasonable.

What am I missing?

(edit: this is meant for IC roles rather than management)

I've updated the profile to include "No FAANG companies" because I'm pretty sure I would not be a good fit for them but that didn't stop their recruiters from pestering me.

For my LinkedIn I listed minimum requirements and it saves me 100+ inquiries per week but I still get a ton of irrelevant jobs.

I took a different offer in the end, but the recruiters reach out every few months for a chat.

(This is a massive oversimplification, of course, but you get the idea.)

Remember reading Bentley's Programming Pearls and fairly sure that's what he starts Sorting section with a short implementation of

Just like I don't think the CA bar should have a pass rate in the 20% range, I don't think coding interviews should ask riddles that are nearly impossible to solve on the fly unless you get lucky and memorized THAT riddle in your studying.

One such candidate I interviewed seemed like they'd be really great for the role: PhD in graph theory, publications, projects listed on the résumé, couple of different programming languages (including ones we used). To me, this person's résumé screamed "solid mid-level developer." I would have probably been willing to pass them at a junior level, had they been able to perform at that level, though.

The interview itself was a pretty familiar story. For the technical portion, I introduced the problem (not a LeetCode-type problem, a more practically-oriented problem), we talked about requirements, drew some stuff on the board, and then got to coding.

I had a feeling when we were going through the requirements discussion that this might not go as smoothly as I'd hoped, but I pushed that feeling aside and did my best to let them shine.

We let people code in any reasonable programming language, but they must write actual code. They can fill in stuff like dummy helper functions, if necessary, but we want to see some kind of running, syntactically correct, and, preferably, at least lightly tested code.

They chose to code in Java, which, while not a terrible choice, seemed to me kind of like they were just handicapping themselves when stuff like (IIRC) Javascript and Ruby were listed on their résumé.

To make the long-ish story a bit shorter, we muddled through trying to implement the requirements we'd talked about earlier in Java, meanwhile the candidate was showing me a distinct lack of familiarity with basic facilities of the language, such as "what sort of methods do lists have that might be helpful here?"

Needless to say, this person did not pass my interview, and we did not end up hiring them. But, I really, really wanted them to succeed. Like I said, on paper, they look great. And, I'm sure they could have gotten through a culture fit interview just fine. I'm just not sure how well they would have done on our team, working on our rather large, pre-existing, and somewhat crufty code bases.

If you can figure out a good way to automate the task of "filter these developers down to the ones who can write some semblance of code," in a way that goes deeper than just "Write some code and run it against our automated test cases," I'd like to hear about that. And, I'm not doubting that it could be done, in theory. For instance, maybe something like the engine behind GitHub's CoPilot could provide a way to analyze and grade the candidate's code on things like style, testability, test coverage, modularity, &c.

But, AFAIK, there's nothing like that out there now, so, a structured process consisting of ~1-hour technical interview sessions, one-on-one with the candidate, attempting as best as possible to simulate the real work environment, is about the best I can think of.

But, here me out here:

Suppose the CA bar exam were changed in format to be more like a SWE interview. That is, instead of being a 2 day affair consisting of 5 essays of 60 minutes apiece, a 90-minute performance test[0], and 200 multiple choice questions, let's shrink it down to a format that fits in one day and under 5 hours.

Now, let's nix the performance test right away, because it would be incredibly difficult to shoehorn in any kind of real, practical task in under 90 minutes.

According to [1], it looks like 100 multiple choice questions are allocated 3 hours worth of time. So, basically, our cut down format could be 100 multiple choice and 2-3 essays. So, that's about half the amount of testing that's done currently, more or less.

Now, if you have half the amount of testing available, you have a choice: you can cover half as many areas of law, you can cut down the depth of coverage in each area, or some combination of the two. In any case, what you've done is increase the variability in the test. In other words, passing is now more likely to be influenced by exactly which version of the test you get.

If there's an area of law you're weak in (say, bird law), it might not even be covered. Conversely, if you're a real expert in bird law, you won't get as much of a chance to shine in the new format. That means our new format both allows somewhat more marginal candidates to pass, and gives up some sensitivity in detecting people who are really, really good.

So, in summary, the new format omits any sort of practical task, increases the variability of the test, increases the probability that marginal candidates will get through, and decreases the ability to distinguish truly excellent candidates.

Seems to me that's sounding more and more like a typical day-long SWE interview, isn't it?

---

[0]: Interestingly, this is essentially a work sample test, which is the thing proven to correlate best with job performance: https://en.wikipedia.org/wiki/Performance_test_(bar_exam)

[1]: https://www.tjsl.edu/academics/bar-prep/california-bar-exam

Of course prospective lawyers should clear a certain level of knowledge before they are allowed to practice, but the passing rate is puzzlingly low. CA law schools, as a group l, really only prepare such a low number of their students to practice?

The bar is hard for CA because we go to school for the theory and then no one really teaches you the formulaic way to answer bar essays. Add on top of that it's a lot of memorization and study. Most students I went to school with had egos and thought they had it in the bag only studying on the weekends.

also, gatekeeping has a negative connotation because it is usually used when it seems unwarranted. If only schools with high quality education are passing students, that doesn't seem like a problem with the gate, but instead with the other schools. I absolutely want lawyers and doctors to be required to pass certain criteria.

Holding this kind of bullshit interview isn't a bad idea if you're a megacorp with chaotic teams and fourteen levels of management hierarchy where people spend most of their time on infighting and career building. You need people who jump through hoops, because successfully delivering most projects in these environments is 10% difficult technical work (which the good engineers can handle, and you can typically hire enough of them via recommendations), 40% YAML-poking bullshit work, and -- optimistically -- 50% jumping through all sorts of technical and non-technical hoops, most of them self-inflicted.

I navigated this kind of process successfully early in my career, and the only thing that made me more miserable than interviews like these was the work I got to do afterwards, after accepting the offers. Once is happenstance, twice is coincidence, three times is probably just how these things are -- I'm now pretty convinced that the quality of the interview is (barring statistical accidents) highly correlated with the quality of the actual position.

Can you give some examples?

I can give one example, I guess, since it's been a long time, it's a very common scenario, and this is an internal thing, so anyone who can identify it is either already there (tequila's on me this evening, mate) or has worked there before (I'm still up for tequila).

The buggiest component in one of the projects was the testing framework. It was a home-grown testing framework, incredibly slow (think "takes about 5 seconds to send a 20-character string over a 115200 UART line"), and it had a huge bug backlog.

Virtually all bugs got closed with WONTFIX. The person who'd written that monstrosity had made it up the management line, and his minion was now in charge of the team that allegedly maintained it. Since all sorts of bullshit metrics like bug fixing rate and total number of bugs and whatnot came up during quarterly operational reviews, a low bug count was nice to have. And since no customer ever complained of a bug in the testing framework, for obvious reasons, all that was required to keep the bug count to zero was an understanding between these two guys that all bugs would just get closed.

So you had to spend a few hours finding creative workarounds, since the two-line fix you'd come up with in five minutes would never get applied. And I mean creative. We had code that literally eschewed non-functional framework code by exploiting a race condition in its code in order to do RPC calls without going through the buggy framework code.

Now imagine you have to do something like this for every single task you do and you have a pretty good idea about how it goes. I mean you technically spend 100% of the time doing what you're supposed to do, but only about 30% of it is actually what you're supposed to do, everything else is mostly fighting cargo cult practices, senior engineering ego, and management disinterest.

I got the offer and took it because I do contracting and had just unexpectedly gotten a contract canceled early, so I was unemployed and have a family. At the end of the 3 month contract they offered me a full time position and I declined.

Especially when I'm not sure the Graph approach is the right one, it's easier than either

1. Getting the Boost.Graph in the source tree to actually compile

2. Dealing with the bureaucracy of getting some, other, Graph library requiring feats of compilation possible for mere mortals into the source tree

This is, however, not something I do from memory - much like the ancestor comment I see the relevant skill as closer to recognizing positive-weight Shortest Path and knowing you want Dijkstra than being able to write Dijkstra without wifi

I didn't study CS at college, and I've never needed to write a sort on the job, so I've only ever written sorts in job interviews! I think I worked out a basic bubble sort, and told them it probably wasn't the fastest way of doing it but that it'd do the job.

There were two - I pointed out both and they didn't seem super happy about it. To this day I'm still not sure if one of them was an unintentional error.

You know, if I had to do that for every PR I reviewed, I'd be burned out in no time.

It was a shame, but if I didn't flunk that interview then I wouldn't be where I am now.

This is, I think, the closest thing there is to a universal experience in this field.

Followed closely by "how did I miss that single-character error?"

Any time I’ve interviewed people I’ve made it a point to emphasize that none of my questions are trick questions and if anything is unclear, they should ask clarifying questions. The result? Interviewees are more comfortable and are far more honest about what they know, what they don’t know, and you get to see a glimpse of what they’re really like.

It is fine to say 'I do not know'. It is fine to guess, but if you do, I would prefer that you tell me it is a guess.

If you want any clarification, I will try to provide it. If you don't recall the exact signature of a library function, ask me. I will note down what I said and not hold incorrect nformation I give against you.

Any questions before we start?"

That doesn't mean you can't debug. You can step through the code step by step in the debugger, or add more and more logging around the bug, until you figure out which expression has a meaning unintended by its author. But that means that, if we're working in a language someone knows well, you're likely to spend hours debugging a problem that they can just see immediately when they're reviewing a merge request. That's an orders-of-magnitude difference in productivity when it's important for code to be correct, precisely due to what you call "precise syntax memorization".

Most of programming isn't writing code. It's reading code.

It's possible to go overboard with this. There are other skills that are more important than being able to look at some code and immediately see what it means. There are excellent programmers with severe dyslexia who will just never be able to do this. But it's foolish to think that gaining this skill is "a waste of time" for those who can.

There are languages I've used where I don't know the syntax that well. PHP, Ruby, x86 assembly, OCaml. There are languages where I know the common syntax well, but there are plenty of obscure corners of the syntax that I don't: C++, Perl, bash. But I regularly switch between C, Python, and JS, and I'm pretty confident that I know their syntax, as well as numerous other languages like Tcl, Lua, Prolog, PostScript, and arguably Scheme and Elisp, which I don't use regularly but still wouldn't have any trouble spotting syntax errors.

Luckily I never fucking ever have to do that in my actual job. If I did, I might well get good at it. Since I don't, I... don't. I also haven't gotten much better at driving semi trucks or framing a wall, in my over-a-decade career writing software. Go figure.

Last time I saw one of those test-ish pieces of code, though, an IDE+static analysis would have caught about 1/2 of the problems; the other half required actual thinking (not statistical pattern matching, aka AI): "don't trust user data, that should not go there even though the call signature matches, you're holding it backwards."

Good type systems do this, although it's beside the point.

The point I was making is that the IDE remembers unimportant things so that my only concern is the actual thinking part. It abstracts away the minor and sometimes very important syntax differences.

IDE tools are great performance enhancers, but they can also be crutches.

I would always expect a professional software developer to be able to parse some code on a page and point out its syntax errors (as well as suggest edits).

edit; here I am thinking about something more substancial then just a missing ';' or a lack of a closing "

    bool x = false;
    x ||= something();

How many do remember exact operator precedence rules for all those languages, when in practice you may need just the basic ones and use () to work around the lack of exact knowledge.

Also which version? Something not working in PHP 7.3 may be ok in PHP 8, but company wants you to code in PHP 7.3, or ES5. In practice you get quickly acclimatized to any of the languages you know after working with them for a few days or a week, but good luck remembering exact rules of any of them at any given time when asked.

And yes, I work in multiple languages.

Performance issues, on the other hand, I can see accidentally arising.

Where you want to make sure a pointer you've been given isn't null before you try to dereference it. Without short-circuit, this becomes a segfault.

I've never seen someone use an "or" pattern in a non-confusing way.

That is, I've seen "and" patterns that act as a safety/early out. I fail to see the use of an "or" pattern where you want to stop executing if the first value is true (ignoring using nots to invert the logic of an and pattern, and I would criticize that).

They do exist in other languages. ES has it in exactly that form. Java has it in the |= form, not to be confused with the bitwise OR of the same form.

Whether or not these short-circuit is not all that interesting for most boolean logic. (Though it can be useful to know if these are used as hacky error-handling and default-setting. It depends on how you read "something" whether or not that's going on here.)

Where and when? Because I was the only person alluding to it, and the replies to my post.

> Whether or not C++ has them (it doesn't) is exactly the point of that code snippet.

C++ has boolean assignment operators that operate the way that the code is clearly meant to operate. That code does not have have a valid one. Where do you think Java got them from?

The words "boolean assignment operator" are in the first sentence after the code snippet in megous' post.

> C++ has boolean assignment operators that operate the way that the code is clearly meant to operate. That code does not have have a valid one. Where do you think Java got them from?

As far as I can tell the |= operator in C++ is the same as in C, i.e. a bitwise OR operator. It works for booleans due to their bit pattern, but it's not the same. My C++ knowledge is extremely limited, so I looked it up and I may be misinformed though.

Java's |= is different for ints and boolean. There are no bitwise operators for booleans: bool1 | bool2 is a strict logical operation (that doesn't short-circuit). bool1 |= bool2 is a logical operation that will fail when other types are mixed in. int1 |= int2 is a bitwise operation. Java does not have a short-circuiting ||= operation (but ES does).

For the most part these differences aren't that important, but they do trip people up when switching between languages.

This is true.

> It works for booleans due to their bit pattern, but it's not the same.

This makes less sense. A bool in c++ is just an integer value with few inputs. It is, as far as I can tell, the exact same.

> Java's |= is different for ints and boolean.

Java has bool and int as different types. There were versions of C++ compilers that just straight out had bool as a typedef of int.

https://www.w3schools.com/cpp/trycpp.asp?filename=demo_oper_...

||= will not assign anything unless the LHS variable is falsy. It will not even evaluate the right side unless LHS is falsy.

|= will work the same in C++ and ES mostly (depending on types)

The fact that the ||= operator short circuits makes sense from an optimization point of view. I will maintain that there is no good use for the correctness (as opposed to optimization) of the code to depend on that feature.

For skilled, experienced programmers, most mainstream languages become an implementation detail. You have to spend time learning idioms, footguns, and generally the way the language manages memory, but you absolutely can be great at 7 languages because they fundamentally do many of the same things.

I haven't hired people based on "their stack" in a long time, and it's been completely fine. Someone with skills can quickly learn your stack and be productive in it. I personally jumped on a project as a coder a few years ago having never written C# before, and I was productive in about a day. All the concepts were familiar, and the stuff I had to learn was mostly syntax.

   > All the concepts were familiar, and the stuff
   > I had to learn was mostly syntax.

If you can be productive in about a day, please explain why a pilot gets ATPL (airline transportation pilot license) after a minimum of 1500 hours of flight. Also please tell if you would board a plane where the pilot has 100 hours - that's an awful more than a day or even a week.

99% of the pain in a new language usually ends up being the (often god-awful) tools, platform/SDK bullshit, and learning where the clearest path is incorrect (no no no, the official docs and tutorials say to do it this way, but everyone who knows what's what actually replaces that entire part of the language/first-party libraries with this other library developed & open-sourced by some other company, since the official way is obviously so terrible, and you just have to know that, or notice by reading other people's projects—ahem, looking at you, Android). The language itself is typically nothing.

This has worked out fine for me. It does mean I've gradually grown to hate languages that lack static typing. I don't want to remember or look up things when I can make a quick note and then let the computer remember or look it up for me. I thought that was kind of our whole thing, no? Having computers do stuff for us, when they're able?

Modern development requires juggling too many technologies for most people to specialize in a single language unless their career goal is to niche themselves to that language.

So at least 4. If you combine webdev with something low level/embedded, you need at least one systems language, so you're at 5 languages you need to be proficient in.

Add one hobby language or a second web backend or systems language, and you're at 6 major languages.

7 is a lot. But 5 is plausible to be proficient in for someone who switches between webdev and lowlevel stuff to not burn out, or has a FOSS hobby.

Also proficient != expert. I met enough developers that were brilliant in their work to be convinced that 5x developers are not a myth, but they are real, while rare, occurrences. For me a senior developer in X knows the ins and outs of that X to the level that his code is an order of magnitude better in term of efficiency, performance, productivity and security. A regular developer can be just proficient, but it is not what I wrote about.

Yes, they are absolutely crutches. All great tools, libraries, and abstractions are crutches. I want programming to be easier for myself and my employees.

The only problem with a crutch is that you might end up not having it when you need it. That's not an issue in this case.

> here I am thinking about something more substancial then just a missing ';' or a lack of a closing "

The original example that I was responded to was about an interviewer who expected their code to compile. That would include incredibly pedantic things.

For example, if you're the kind of person who uses single quotes in JavaScript and then you're suddenly writing a different language where '' is different from "" and `` and $"" and whatever, you could easily make an unimportant mistake that prevents compiling.

Having said that, if someone came up with an interview test which used especially esoteric parts of the language in unconventional ways and then asked to spot the errors, the could be a dubious question.

When did I claim that the IDE absolves you of needing to know the syntax?

It doesn't. But between autocomplete, hinting, linting, and any other static analysis, it makes it close to painless to switch between languages without making horrifying mistakes. The top-tier JetBrains IDEs (IntelliJ and Rider come to mind) will even tell you ways to make your code more efficient or modern, like changing a bunch of if/else to pattern matching.

Why should I have to remember the full truthiness table of JavaScript? Why should I have to remember what all the different string delimiters do in every language? It's not important. My IDE can (and does) know that I'm trying to do some kind of string interpolation and will just fix it for me.

I remember thinking what you're writing about there, that there's a lot of people to be filtered out, and that good candidates wouldn't be offended.

So we had this simple two-part quiz question for people, starting with "what is the expectation of a dice roll?". Amazingly a lot of people can't figure this out.

But also a lot of people know the answer immediately and will wonder WTF you are asking such a simple question for. I remember this one lady who interviewed with my firm, the look on her face when she realized we weren't asking anything complicated. You could just tell she thought we were a bunch of amateurs, and she'd better be on her way to see some other proper hedge funds.

But I think the point in this thread is the Google question about Linux inodes was actually part of a pre-screen interview done by an outside agency.

In contrast, I've been stuck in a giant multi-language integration-fest, and... well, there are definitely languages on my resume that I would not be comfortable being pop-quizzed on, simply because I've been using others for the past two years.

_Can_ I make sure the code is 100% correct before even compiling? Sure, but I'll spend an hour checking every detail, while intellisense does it while I type.

Let's turn this around: if I were interviewed by someone who flagged down my code for missing a #include or lambda capture (both very easy mistakes to make), I'd know that the people I'm interviewing with are idiots with no understanding of the thing they claim to be testing me on. Would I want to work there? Nope.

There is a difference between "find the errors in this provided code" and "write code on a whiteboard with no errors". At no point was I talking about code you wrote.

Also, it's an aside, but I find people who can program well write excellent documentation for the users, if what they are writing is an API. Of course, they are not the best at explaining the steps in a GUI, but that probably has less to do with communication skills in general and more to do with the difficulty understanding how they perceive the problem.

But yeah, I think I've seen questions like that for an intern positions. It's basically a "have you ever seen this language?" to weed out people quickly.

It's not that. People who constantly use multiple languages in an IDE will not be able to point out most syntax errors outside the IDE. So it's not a "have you ever seen this language" filter.

But I'm curious what position would this question make sense?

I was once asked a similar but better posed question: I was given some code and asked what I'd point out if asked to do a code review on it.

And the code had loads of things wrong with it, from bad variable names and incorrect comments, through unit tests that didn't have any assertions and loops that weren't actually loops, all the way to choosing a non-secure random number generator in an application that needed a secure one*

In other words, it was a test of my ability to code review, with some glaring issues to give me some easy marks and set me at ease, and some subtle issues where talented people could really set themselves apart. It was fair and relevant to the job because I was presenting myself as a senior programmer with lots of experience doing code reviews and coaching junior developers.

It's possible trinovantes's interviewer intended to give the same sort of test - but either didn't explain the question clearly enough, or trinovantes misheard or misremembered.

* A bug right out of puzzle 94 in 'Java Puzzlers'

At a different interview, this one with Microsoft, the lead engineer showed me a page of actual code from their app and asked me what I thought. Luckily the bug instantly leaped off the page to me, although many people would not see it. That was a good way of proving that I have a useful ability, and a better use of time than whiteboard coding or quizzes. I got the job and they were glad they hired me.

Coding is a mostly solitary activity that you probably don’t want candidates spending more than ~60 minutes on, ideally on something that resembles the actual day-to-day instead of sucking all the Leetcode possible out of them. Even just quickly pair programming on something nets you more data points in the same time.

I had 10+ years experience in C++, I thought it was completely reasonable. Especially compared to their later questions around something along the lines of finding a shortest path in a tree. I interviewed there a bit before their process was so well known to be game-able, I went in with zero study time on obscure algorithms outside knowing the O(N) of the most widely used, and certainly did not practice actually writing or interacting with trees and such.

There was a function with a syntax error, that also returned a pointer to stack memory, and made some logic error where it assumed a class with no vtable would be polymorphic.

Seeking out problems and errors can be a good conversation piece. Hopefully you get to hear some anecdotes, prod the taste in style and how well that that taste might play with others. The interview situation isn't easy for anyone, and anything that can if something is even remotely qualified helps.

"What does this code do?"

That was a pretty easy one to figure out, it pulled coordinates from a database table, and then it stepped along all the lines trying to find the longest one (they were a metal shop).

"Do you see any evidence that this code has been optimized?"

That was the dumb question.

The questions go like,

What is the output of the expression below?

    int i = 10;
    ****++&&*+p;

Not sure how this measures language proficiency.

> int i = 10;

> **++&&*+p;

> Followed by a myriad of options. Including things like Syntax error.

I consider myself fluent in C and to a lesser extent C++ -- that's a Syntax Error in C, at least.

This isn't a particularly difficult one to spot, but I can understand how it would be if you weren't very familiar the language.

Eventually your eyes will give being a lexical analyser.

(/me runs away)

[1]: https://www.ioccc.org/

In this interview, I would have liked to receive two code examples (that might contain errors) and discuss benefits according various objectives.

If the interviewer makes it clear that pointing out syntax mistake is not rude, I could mention them in passing. This demonstrates not only attention to details but also decorum.

Recruiters need to understand that these kinds of processes will often filter out the wrong people, such as those skilled enough to be able to pick and choose.

Candidates often have to call out nonsense otherwise it may never be called out. Processes need feedback to adjust and adapt, otherwise they'll typically continue with momentum alone.

With that said you can give feedback in a polite and professional way, you don't have to be arrogant about it. "Based on the questions, it appears you're searching for these specific abilities which are often attributed to a junior role, so I believe I may be a mismatch for this specific role. I'm going to politely withdraw my continued involvement in this process. I appreciate your time and interest and hope you will contact me if a more senior role is available." Or something to that effect. You don't have to be arrogant to give feedback.

If you were like "what, this is ridiculous, what am I am an intern? Good luck filling this trash position!" And then walk out then sure, that person clearly had some anger management issues.

Interviewer: How would you reverse a string? Me: boggle Any language I want to use? Interviewer: Yes. Me: Okay, Ruby. "somestring".reverse! Interviewer: boggle Me: I don't think we're aligned on what this role is. says thank you and leaves

Interviewers need to understand what they are interviewing for.

If OP was a dick about it, then yeah, it serves to filter out an arrogant assbag.

But if OP simply explained that the interview led them to believe the position was a more junior/entry level than they were expecting, that seems fine. Further, to even explain that the interview process seems to just be a checkbox process seems fine; if you work in a critical thinking/creative role, checkbox culture is an absolute brain drain.

Getting that out in the open, in honest and respectful terms, is a fine thing to do. Why wouldn't it be?

Further, any hiring institution that feels the need to build in 'tricks' to filter people out of the interview process is toxic. Even if the people they're filtering are arrogant assbags.

If you I consider that arrogance, then so be it. I consider it not taking jobs that'd make me miserable, because I don't need to.

A prospective employer doesn't have a right to have me bend over for whatever process they'd like.

It very much depends on the code - but I was genuinely surprised how many applicants, claiming to be fluent and applying for a senior developer position, had problems just grokking what the code did (a while loop reading from database).

That’s true whether I’m an author writing in German, a newscaster reporting in Italian, or a programmer coding in C++.

What you say applies if you're hiring people for their first position at that task (e.g. the author writing their first book in German or a newscaster who has never done reporting in Italian professionally). If you're hiring people at some hypothetical "level 10" then your interview needs to discriminate between "level 9 or less" people and "level 10 or more" people, but asking them to assert that they meet "level 1" implies that they might not, and that implication is literally insulting.

Switching part of the interview to be in Italian or German would not be seen as disrespectful, right?

It’s interesting that some find the coding equivalent insulting rather than merely a bar pointlessly laid on the ground to be stepped over.

A bar pointlessly laid on the ground to be stepped over is reasonable iff it's you can just quickly to step over it - but if they ask the candidate to waste half an hour to prove their capacity for stepping over bars laying on the ground, that is disrespectful of their time.

For programming, a trivial short task (e.g. fizzbuzz) is appropriate but a trivial long task is appropriate only for junior positions but disrespectful for senior ones - ask something that tests whether they're capable of something serious, because passing the trivial task can't be sufficient anyway.

If there is an error, I can quickly figure out what that is and what I should do to fix it. I would know why the error occurred.

Beyond that deliberate interview practice is the only way to get a lot of interview questions right.

(Admittedly if the PR doesn't build why are you reviewing it but whatever)

And you are having a CI build and unit tests in place. If it doesn't compile or a lot of tests are failing, a sane person won't even bother to review a pull request.

If you check out the branch in your IDE, is there a way to have it highlight the changes in the branch you're reviewing? Or do you need to reference the output from `git diff` or the github PR view?

1. Open Source control window

2. Checkout the PR branch

3. Open the branches listing panel in the sidebar.

4. Mouseover the target branch of the PR

5. There's an icon that looks like two nodes with arrows pointing between them. Mouseover text "Compare with ...". Click it.

6. The search and compare panel in the sidebar has a listing of files changed, you can click a file to get a diff view.

There are gitlab [1] and github [2] extensions which streamline this workflow if your code is hosted on one of those services, and let you leave comments in editor which show up in the web UI.

IntelliJ has support for display a diff for a branch or github PR built in [3] but I hate their diff modal view.

[1]: https://marketplace.visualstudio.com/items?itemName=GitLab.g...

[2]: https://marketplace.visualstudio.com/items?itemName=GitHub.v...

[3]: https://www.jetbrains.com/help/idea/contribute-to-projects.h...

https://github.com/features/actions

I don't see a lot of focus on the hide-the-ball aspect of interviews, but is something I've experienced a few times and bothers me way more than anything else.

The clearest time this happened I was told the company was big on pair programming, so I'd be doing a pair programming session. It turns out they meant I'd be tested on whether I could finish a coding exercise in the allotted time with someone watching. There was zero "pair programming" of any kind involved and time spent on collaboration counted against me.

HR wished I had gotten a different interviewer.

But I kept being approached by Google recruiters, kept recounting what had happened last time and asked if I could expect better this time. None could promise things had improved, and a few did express that kind of frustration.

This was years ago now. Could have changed of course. But I started telling them to go away.

Now I'm getting an email every month from FB, and about ready to do the same with them.

The last time I passed the first and second rounds and Google let me languish on the third round for a month. I still probably would not have made it through, but it was surprising that my candidacy was dropped like that.

Yet, you could expect one of those in almost any interview process. Seemingly smart people are ready to jump on bandwagons, too.

Microsoft started doing this, realized it was a bad idea, and stopped before Google even really started this practice. So Google is as guilty of bandwagon jumping as everybody else here.

If the interviewer cannot do this, how is he going to judge the result? Does he know how to run a compiler? Does he know how to run the code? Does he have the skill to judge the output? Is it really a good policy for a company to discard a possibly excellent candidate that just missed something silly that would normally be checked by a tool while you type?

And if you fail to communicate the requirement for one or the other then you're certainly not a competent interviewer.

You also have to ask what exactly is being tested here? Is it the ability to remember syntax? To remember an algorithm? To improvise an algorithm? To recognise which algorithm is needed?

What, exactly?

In my view, if the answer involves a topological sort the interviewer should know how to solve it and be able to follow and find errors in the candidates code. If the interviewer, knowing the answer, cannot find any issues then surely the code is fine (for code written in an interview)

Actually hang on, I'm editing this to be slightly meaner. Your whole take that doing this is a sign that she's either an idiot or on a power trip is a very familiar thing that people say about women in tech and I'm honestly tired of it, because I can see myself doing exactly what she did and I don't like it when people say those things about me. Please don't do that.

You just can't tell someone to type code into google docs and expect it to just work and worst off all judge a persons skill on this basis. It takes minimal experience of programming to learn this. Hence all the jokes that people are surprised/suspicious when their code runs after first compile.

If you disagree with this you could have provided any sort of counterargument. Instead you took this weird "women in tech" angle. Wrong is wrong, interview here was wrong, gender did not play a role.

Or they can run it, asking the real arbiter of truth whether it works or not.

Of course, "whether it works" is merely one (very important) metric of quality.

Can you work with this person? Can you collaborate to write code, or will it be a daily struggle?

Whether the code actually builds and runs after the hour is up does not help answer these questions; it is arguably the least interesting part of the whole process. The time limit is artificial; if all the other things align but you didn't happen to get it working in one hour, you'll likely have got it in two. If they don't align, you'd likely never have got it.

This is like writing code in notepad and creating a PR without building or running it to test once. What is this testing? There is no real world scenario where you are expected to work like this and for a good reason.

If you're interviewing for specific language skills, then what you say is clearly true.

If it's a general coding skills interview, I invite the candidate to code in whatever language they like. More often than not they choose a language I am sufficiently fluent in to follow along. In rare cases I need to ask them to explain a thing or two. In very rare (but generally quite fun) cases they choose a language I am totally unfamiliar with; this tends to lead to interesting discussions.

In all of those cases the full coding transcript is captured and, if necessary, can later be additionally reviewed by/with someone who knows the language well.

P.S. Also, their thought process and approach to problem solving is just as important as the code, and is largely independent of the choice of programming language.

> I thought I was done, then the interviewer said she would go go copy my code and compile it after the interview to see if I was right, which blew my mind.

I've been interviewing at Google for nine years and have never done this. I generally don't think it's fair to ding a candidate for something I don't notice in an interview, where I'm at a huge advantage. If it looks right to me then that's good enough.

But that said, I have often asked questions similar to what you describe. You have to write code in the shared doc because hiring committees want to see it - fair to blame the process for that, but not the interviewer. I actually appreciate this for a couple reasons:

* It levels the playing field a bit, in the sense that code is more objective than an interviewer's notes on how a conversation went (especially for people who aren't native English speakers)

* I sometimes find that candidates who communicate well struggle to turn their ideas info code. Other times, someone's communication and solution are kind of average, but then they use all the little things I like seeing (defaultdict(set), zip, etc. in Python). Lots of people claim to be very experienced programmers; seeing how comfortable and fluent someone is when actually writing code is a strong signal. If you don't like the focus on data structures and algorithms that you haven't thought about since college, you should probably appreciate the coding part.

Like thats how I would feel writing code into freakin google docs during a job interview...with Google.

Tie 1 arm behind by back as the synax goes wonky, I'm fighting the spacing, etc. etc.

Or put mario Andretti into a ford focus, then test his lap times, with 0 warning.

Yuck.

If you're a world champion, you would probably enjoy an exercise like this.

This lady pilot sure did: https://www.youtube.com/watch?v=5KiC03_wVjc

That said, if a person is being considered for a coding job, wouldn't that person want to demonstrate their coding abilities? I mean, assuming they're actually good at coding?

In interviews, I much prefer a concrete problem to an abstract one, and coding problems are typically far more concrete than most other subjects covered in SW dev interviews.

You may usually write code in a terminal based programmer's editor (vim, emacs, ...) and realise the code you've just written is not quite right. You want to delete the last two words. There's even a default and handy keybinding for doing that.

So you press ^W twice.

If you are gonna require a candidate to write compilable code at least use one of the many tools designed explicitly for coding interviews.

I think I’d rather use straight notepad than google docs.

Except that writing code in a Google doc on the phone doesn't in any way resemble the real "playing field". I grant that it's level in that everyone faces the same constraints, but to write code in a gdoc with red squiggly lines underneath every keyword, automatic capitalization after a dot, and variable width font? How does that give you any kind of useful assessment? It's like handing a butter knife and some twine to a doctor and saying "here, show me how you'd stitch up this wound".

When viewed in that light it actually doesn’t seem quite as bad.

When you have 3 interviews per week for a prolonged period, you, as an interviewer, are not going to do a stellar job every time. What's worse: you will develop a routine and it becomes very easy to give candidates that do not fit your routine a lower grade. It takes effort on the interviewers part to recognize talent that perhaps doesn't fit your routine or your expectations. If you are not going to end up being a bad interviewer you also have to try to relate what you see in interviews to what you know about work.

For instance I _never_ asked people to code live (mostly on whiteboards back then) because it just isn't a relevant exercise. And I was kind of horrified at experienced interviewers who asked people to code and then got obsessive about small details that the tooling would have taken care of. Absolutely pointless.

The only piece of advice I found useful from the interview training was this: this is the candidate's big day. For you it is a chore, for them it is their big chance. Keep that in mind and respect it. I kept telling myself this for every interview - and some days I felt really terrible because I wasn't properly prepared.

The other thing that horrified me was when we let inexperienced people who had been out of school for less than a year interview people. These interviewers barely knew how to write software themselves, and they'd get even more hung up on irrelevant stuff because they simply had no idea how to be software engineers.

I doubt that I would have done very well in those kinds of interviews because this isn't how I work and it certainly isn't how I teach people to do problem solving. Problem solving requires more time because any even mildly tricky problem worth solving tends to have a lot of facets far beyond picking an algorithm or knowing how to code it up. That's the easy part because for that part, you have books, papers, tools and other people to seek advice from.

Junior programmers right out of school with no engineering experience have no business interviewing developers. They make poor and overly judgemental interviewers and only rarely are able to spot talent if it doesn't fit their template. They also aren't going to fight for candidates that may not fit the imaginary template, but have some special gift because they are junior programmers. It takes a certain amount of balls to say "I know you think this candidate is rubbish, but I see something here and I don't care what you say, I am going to insist".

(btw, statistically, this used to be a good predictor for later success: candidates that were somehow "controversial" in that they didn't make the grade with some interviewers, but displayed something that made other interviewers fight for them)

Thank you for being kind and respectful.

We do NOT require you to produce token-by-token perfect code, and nowhere in the process will I ever have to give feedback on whether the code produced was actually valid. So maybe you misunderstood your interviewer's intention, or something else went wrong, or maybe they were just joking. But it's not the interviewers task to copy code into GCC and try it out, that'd be a huge waste of time. So let me be very clear and explicit: no-one gets classified as "no hire" because they've forgotten a semicolon somewhere. You messed up somewhere else.

With that said, if a candidate says they can write in a language, we expect them to know the language, its idioms and at least parts of the standard library. Not every nook and cranny (e.g. I'd often instruct candidates "just pretend you have some library that implements a heap, and invent an API for it, I don't care about that part"), but if you call "strlen" in the termination-condition of your for-loop instead of before, or do other stuff that shows you don't know the language well, that's a red flag. After all, we expect you to be able to write production-level code that servers billions of users.

“We do NOT”

“You messed up”

“let me be very clear”

“whether you’re able to code”

These types of statements come off as patronizing. If your whole process sounded like this, and if you somehow actually do represent a majority of Google interviewers, then it’s no wonder so many folks have a distaste for the experience.

It depends on who you get as your interviewers, so generalizing isn't really useful. Some interviewers can't relate the interview to what it is supposed to tell you. I've seen that in inexperienced interviewers and I've seen that in very experienced interviewers who had enough GOOG stock to buy a small country.

If the interviewer can't think of a better way to test candidates, that's on the interviewer. Not the candidate.

How much money can you bet that this never or does not happen? If you can't put your money, I find it difficult to take this seriously.

> We don't usually care if you don't know find the perfect algorithm (unless it's for a senior position)

The unless clearly means that you "do care".

I can see this interviewer writing feedback like "Well, this candidate didn't come up with a perfect implementation of Dijkstra's shortest path algorithm, but its OK, they were pretty close. Oh wait, they are a senior engineer? Nevermind... they should have really mastered these algorithms while building CRUD APIs all these years."

And therein lies the problem.

As far as algorithms/data structures, most of what we do on a day-to-day basis is more about selecting the appropriate data structures and algorithms that fit the problem, and assembling them together correctly. I've never needed to code a hashtable, but I need to think about their characteristics and whether they're appropriate all the time. I've never written a btree, but I do understand database indices pretty well. So in my view, asking candidates to code up these things on the fly in an interview with no reference materials is a total waste of time. What matters more is if they can correctly apply them. If you're really sure someone needs to write these things, then a more realistic move would be to sit them down with a standard textbook or paper and see if they can get the code working.

> After all, we expect you to be able to write production-level code that servers billions of users.

And you do all this through sheer clairvoyance, rather than using tools like unit tests, code review, design specs, etc, right? No? Then why are you expecting people in interviews to do it without those things.

The arrogance displayed in your comment isn’t backed up by the ultra-low quality of product being produced by Google. Something is clearly wrong with the culture and hiring process there. Maybe instead of fretting over people using strlen as an exit condition, you should look for people who can actually produce software people enjoy using. You don’t even need to do a coding interview! Just browse GitHub for people you want to hire and get to work convincing them to join you.

As someone mentioned in another comment, I also never had to actually implement a sort since I left university more than a decade ago. I'm happy to discuss the different types and approaches to how they could be done - as to evaluate problem solving, logic, and general understanding - but to actually code in an interview...

In a similar way, this week I even jokes on Twitter how I'm doing many improvements and refactors on a codebase, but there's the occasional 'how I do join 2 lists on Java again?' moment when I completely forget something basic.

I also complained that google's coding solution (docs, basically) was a terrible way to code, other companies use coderpad or whatever, but I expect that Google will never change this. They love to hire people who are excellent at coding CS approximate solutions, but have little to no judgement on how to do good software engineering.

  1. You should have been better informed about the expectations of the interview, so you would have had a chance to prepare yourself.

  2. Coding in a Google Doc is a terrible experience. It's a step up from coding on a whiteboard, but that's not saying much. Google has since moved away from both, prefering an online text editor that's not quite an IDE but at least more programmer-friendly.

  3. The goal shouldn't be to write 100% correct code without any compiler feedback. That's insane. Still, there is a big difference between "candidate forgot a semicolon once" and "candidate did not know how to write a for-loop without IDE feedback".

  1. Can they figure out how to solve a nontrivial problem?
  2. Are they able to translate ideas into reasonable, working code?

  1. Those that don't have a clue how to solve it.
  2. Those that recognize it boils down to a topological sort.
  3. Those that recognize it boils down to a topological sort and are able to implement a solution.

> I didn't remember the exact algorithm so I basically had to re-figure it out on the fly

Yes! That was the whole point of the question! You had already demonstrated that you were at least a "type 2" candidate, so now the interviewer was trying to move beyond that and figure out if you were actually a "type 3" candidate. Nobody expected you to have the exact solution memorized, but they expected you to be able to figure it out from first principles.

> I even similarly mention "in any real situation I would just look this up", but that didn't help

That was missing the point, which was to test your ability to actually implement a solution.

In the real world, if you encounter a standard problem (which happens often, like "I need this list sorted" or "I want to put this stuff in a hash table for O(1) access") you wouldn't even look up how to solve it. You would just call the existing standard library function and move on.

Logically, the problems that you end up spending most of your time on are not of the standard variety, and involve actually thinking about how to break down the problem and actually implementing your intended solution. Those are the problems Google needs you to solve on the job. If you can't even implement a topological sort from scratch, why should anyone expect you to do anything more complicated than that?