Google is beginning to show its age, I’ve seen a number of past coworkers get hired there lately that weren’t effective at their job, but I’m sure they crushed some Leetcode.
Yup. Goodhart's law strikes again. I am curious as to what the next as-of-yet-ungamed effective hiring metric is.
See also: college admissions. (No, that kid probably didn't do that volunteering activity because they wanted to help people, but because they were told it was a checklist item, as they were coached through college admissions. And they probably didn't learn anything, and they were coached on what to say about that in the essay. And, normally, actually productive volunteering probably wouldn't have looked like each one of those kids starting their own duplicate initiative so each could spin it as demonstrating leadership. And it's more an option for kids from families well-off enough that the kids didn't have to work jobs that would contribute better to their household income or college expenses. And don't get me started on how the well-to-do decided that "travel" would be a plus on applications.)
Then they give you your real chair.
It is probably realistic to expect someone to write something useful (although possibly small) in three days. It is less realistic to expect someone to write a useful component integrated in a large system that they have to learn, in three days.
Is that actually true? It's probably uncommon in tech but I understand it's fairly common in creative fields and--while it was a long time ago--I worked a couple unpaid internships when I was in high school in chemistry labs.
A railroad engineer intern can move cars around the yard all day, assembly trains, and other such things. However if they move the car to a loading dock where it is loaded/unloaded that is valuable work and must be paid. If the train they assembly leaves the yard for the next that is valuable work (unless they all make it back to this yard unused).
Most companies choose to pay interns rather than figure out how to not break the rules. A few have a separate intern program where they carefully figure out tasks to assign interns for training. If your intern program did valuable work you should talk to a lawyer.
Not to mention, you'd have to have a precise and well defined notion of what "analytical hobbies" are. Preferably one that doesn't carry an inherent classist[1] (and maybe even racist or sexist) bias. That part alone seems pretty tough, forget the actual definition part.
---
[0]: https://en.wikipedia.org/wiki/Eight-legged_essay
[1]: https://hbr.org/2016/12/research-how-subtle-class-cues-can-b...
That's the truth for me, I was shocked some of our worst* interns went on to work at Google after the summer.
* Unproductive, slow, needed lots of hand holding
All interns need handholding but some need 10x more than others. And yeah, they often end up doing another internship or so at a faang.
Alternatively maybe those people were just unmotivated to work for you and spent their internship preparing for Google.
At least in our place, we do not really expect them to be productive in the usual sense, but they definitely have ample opportunity to give us a feeling of their place on the bell curve.
(as an anecdote: We currently have some 17 year old apprentice which puts some of our seniors to absolute shame, myself included. The amount of motivation, learning hunger, quick thinking and ability to ask the right questions is really astounding, especially given that he essentially just started and has next-to-no education in our area to speak of as of yet)
Too be clear, I’m not accusing you of being lazy, but your description seems to apply to me and I consider myself lazy.
At this point the whiteboard gauntlet seems more indicative of a lack of critical self evaluation than anything, “we only hire the best because we’re the best rah rah”. Miss me with that shit.
Google's search results aren't because they suck, they're doing it to get maximum money. This is about getting maximum cash for minimum overhead. The Goog is a publicly traded company, they're about selling you the crappiest product at the highest margin. Until someone can come and seriously eat their lunch in search, it will never get better.
To paraphrase Cory Doctorow, "en-shit-ification" is explicitly the point.
They hired the best and the brightest not to make search better, but to find ways to optimally increase the stock price; good products are entirely tangential to that.
Google also operates with their own sets of constraints. What works for them isn't necessarily what works for different types of companies.
Isn't the story more complex than that? I certainly didn't just get leetcode questions when I interviewed there, and when I asked, the interviewer said the group hadn't been happy with hires that only knew algorithms.
I had something like an 80% offer rate because of this. Ironically I didn't actually get a Google offer though (not sure why, I did solve the problems).
2. Did you get an offer? Did you accept?
It's also more common now to have two coding sessions, and even two system design ones, under the rationalization that it reduces interviewer bias.
Google explicitly links to leetcode/hackerrank/etc as suggestions for practice resources in their pre-interview email, implying that practicing artificial DS&A quizzes is now table stakes.
I would be curious as to whether these changes have been data driven, or whether the bar is being increased purely out of necessity to weed out candidates more strictly.
I certainly have seen my share of candidates that do fine in leetcode sessions but then go on to fail in the other sessions for various reasons that allegedly correlate to work performance.
> Google did the research
Did they? Where is the proof? Pot, Kettle, black
But it is hard to say how much leetCode style interviews helped. Post-2015 Google already had quite a bit of momentum.
Cracking the Coding Interview came out in 2008 and had a short section talking about the Google interview process, which I believe was system design + a coding problem that would probably be close to a modern LC question.
I did a fair number of coding interviews (for SRE positions) at Google (I don't actually know how many in total, but 25-30 is probably a safe guess) and, yes, it started with a small problem that should be solvable in well under half the interview time. I only used problems that passed my "is this fair to the candidate" screening (look at problem, try to write a working solution, if that takes more than 7 minutes, the problem is not fair).
The value in the interview comes from (a) listening to the candidate narrating their thought process during the coding, (ii) discussing the solution with the candidate, sometimes in terms of complexity, sometimes in terms of correctness, depending on what is on the board, (3) refining the code written (add more functionality, change functionality).
For a language I knew, I tended to overlook small syntactic mistakes (a whiteboard does not have any syntax highlighting) and if there was any questions about library functions, I would give an answer (and note down what I said, so that would be the standard used when judging) and the bulk of the score came from the discussion about the code, not the code itself.
If that matches what you mean by "LeetCode-style interview", that's certainly been in place since at least 2011. If it does not, things may have changed, but at least the aim wit ha coding interview back when I was still at Google was less "get some code judge, on the code" and more "get some code, judge the discussion about the code". It is also entirely possible that interviewing for SWE positions was different frmo hiring for SRE positions.
Basically the equivalent of the Roman emperor thumbs up/thumbs down except it's a small group and you need a certain number of thumbs up to move forward. The number of people you need Inclined to Hire differs from org to org that use this method.
I worked for Google and at least back then everyone working there could read the studies showing that the interviews worked. The effect was very clear, with there being no upper bar where the interview score stopped mattering.
If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. But I'm not sure why Google would lie about that, or why they would keep the interview format for so long if it didn't work. The old article talking about Google interviews talked about what they did before they arrived at this format, what they do today is the best interview according to their data.
I don't work at Google, so I cannot speak for them. My company does have studies and experiments around interviewing, but again, the metric they use for success is "inclined rate" e.g. doing ABC leads to x% increase in inclined rate. I haven't seen any study linking interview outcomes with on the job performance e.g. yearly performance rating or promotions. In fact many people have requested this data over the years.
> If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course.
This comment seems quite rude, and again, it misrepresents my position. I didn't say LC style interviews don't work, I'm saying that I don't know, because I have seen no data to prove that they work better or worse than other alternatives.
Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.
But that is exactly what Google did, they linked the interview results to later performance evaluations and that is the correlation I'm talking about. The better they scored on the interviews the better the performance reviews they got later. The correlation was very stable and didn't just appear at low levels.
Your company might not have done such a study, but Google did.
> Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.
That is true, the data I saw is nearing a decade old now, but at least the data back then showed that it worked.
One possible explanation of this is that people who did well in the hiring process were valuable employees. Here's another: people who were good at gaming an evaluation system run by Google were good at gaming an evaluation system run by google.
All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.
Note the hiring managers are the ones picking the criteria.
My guess is a dev who passes leetcode but fails to produce is not seen as reflecting upon the hiring manager. It doesn't matter if leetcode correlates or even negatively correlates with the success of the new hire; what would matter is not being able to blame the hiring mgr.
Is high performance making economically valuable contributions? If so, there is huge inherent inequality between the economic value of projects, so the metric is worthless for judging engineering talent.
Is high performance getting good performance reviews? In my experience performance reviews are wrought with favoritism, ageism, and probably other subjectivities that create a very low signal to noise ratio, so that metric is worthless.
Is high performance someone who is good at writing software? There are engineers I know who write very good software but lack the decision making or vision to do so in valuable areas. Thus, another metric is worthless.
The reality is there is no general “high performance”. There are smarter and more motivated people, sure. I think testing for general intelligence, personality type, etc., are valuable. But I also think a lot of high performance is about finding someone who is a good fit for the organization in terms of the individual’s experience and skills and the organization’s needs.
This variance is why it is so hard to assign objective numbers.
Bugs are not created equally.
- assign low effort, high impact, high visibility, low risk projects to the most favorite engineers.
- assign high effort, low impact, low visibility, high risk projects to the least favorite engineers.
Favoritism in toxic orgs is the reward for "loyalty".
Another way is by reorganizing teams, assigning favorites to a team just before the completion of a project. Or assigning undesirable employees to a failed project.
I've heard that claim times and times again.
Often it's from companies believing in "Best Cost IT talent" or whatever the current buzzword is on LinkedIn. Chances are, they never even worked with an SV caliber dev.
We've all seen engineers that are simply running around in circles around others. Just think of John Carmack. If any of these organizations managed to luck out by hiring someone from, let's say Apple or Google, it would most likely change their beliefs that there's no "high performance".
if they are regularly practiced at a company, this means the company puts enough faith in them to overall promote those who deserve it and to decide peoples' and company's fate based on that. Which then makes it quite a decent interview outcome metric.
If on the other hand performance reviews are as bad and worthless as you claim, they should be stopped completely, and everyone running them should be instantly fired as a bunch of ageist favoritist pigs who are scamming the company.
They could have been fooled by randomness, like many "intellectual" types often are.
If I remember correctly, it was actually slightly negativity correlated with work quality because extremely talented competitive programmers would smash the interviews but write competitive programming style code (aka not enterprise quality).[1]
1) https://catonmat.net/programming-competitions-work-performan...
the interview is about the interaction on a CS topic. how do you express yourself, verbally and in code? that's not: can you crank your the highest performance solution in the shortest amount of time.
I can tell you that at our company we definitely dodged candidates here and I was very happy with everyone we hired (but you could call that bias, as I was the hiring manager -- but I also worked withe veryone I hired)
FAANG-like hiring has the question "What info do you get from the 5th interview that you didn't already have after the 2nd?" - for the paid trial, what info do you get by Friday that you didn't have by the end of Monday?
How was it gamed? I wasn't aware of anything. Is this about LLMs being able to solve some of the questions?
Sure you need to understand some basic CS to pass a LC question, but it’s more about recognizing a pattern and having memorized the proper algo for that pattern.
If you were taking an algorithms interview 20 years ago, you’d have to truly know a huge breadth of CS knowledge. But now you can just memorize problems for a few months and never learn the true meaning of “why” a tree is the best data structure in x situation.
It was “gamed” when people learned that you could pass the interviews without truly understanding the material, if you just memorize the rules of the game and can recite them when asked.
It's also not as easy as you describe - just memorizing patterns. Maybe for the easy-medium questions that ChatGPT can solve, but Google stopped asking such questions a decade ago. The algorithm questions that all the tech companies ask today require real problem-solving.
Did they? I remember hearing about a research they did and it wasn't so clear cut to its advantages as a "job success predictor"
oral exam style algorithmic interviews are.
Where would we be without such hiring processes that managed to have such set of genius work on Android.
/s
Everyones approach looks like it works because if everyone is between a 90-110% worker and the team gets along and works together you will get solid results.
Big assumption that performance can be reviewed and scored in some objective way, or if it is that this actually happens. I've seen lots of review processes that pretend to be objective, but absolutely none of them actually were.
Same goes to any subject where a multivariate analysis is almost impossible due to the lack of control over your experiments. I seriously doubt that anybody has any data that could prove anything.
As an interviewer who did this for many companies for a decade, I hire for attitude. I can teach anybody hard skills (software engineering, sql, python, etc.) with enough time but I can't teach attitude.
A person who I can trust because the persistent drive towards goals, can deliver a lot more than a naturally talented programmer with shitty attitude. This is based on my anecdotal data that I accumulated over the years.
Funnily enough this is similar to what the Seal Team figured out about recruiting in a completely different field independently.
That's just part of the process, though. They (and many others) have coding questions, system design questions, and behavioral questions. Taken together, they answer the questions of whether you can write code, design/build/contribute to a coherent larger system, and are someone who can be effective at working with others in a company.
Over time, there's been a massive (IMHO) over-index on the coding portion, since it can usually result in an objectively correct or incorrect answer. With massive numbers of applicants, companies really don't lose out on being over-aggressive with cutting people based on these things. As they tighten their hiring belts, I think this will get worse.
[Citation needed]
(Or you are doing what you say others are doing)
At least the reason why we have Leetcode questions is because Google did the research and came to the conclusion that those are were good at algorithms ended up being more successful at Google, and THAT'S why we are all suffering through LC. But now that LC has been gamed, I would love to see what the results are as to what makes a successful interview.