* query a database
* run a web server, typically spring boot or a C# equivalent
* run some ridiculously large browser framework like Angular or React
Typically the end goal was just put text on screen and you need an entire army of developers to do it. Any suggestions that deviate from common patterns were met with hostility by other developers and ignored by product owners. The name of the game was hiring and firing.
Almost no corporate developers were doing anything that could be called engineering. Most of the people that were interested in actual engineering were writing open source outside of work.
-- CS grads are, on average, better programmers than non-CS-grads.
-- The average CS grad is not a very good programmer.
My company does a standardized interview with a straightforward coding problem that is only a few steps above fizzbuzz level. In just the past few weeks, we've had candidates who:
-- Had to look up basic syntax (defining a function) in the language they chose.
-- Duplicated a reference a few minutes in and could not figure out the problem in the remaining twenty minutes.
-- Couldn't extract the 2nd element of an array of five elements.
These same candidates, on other portions of the interview:
-- Knew nothing about common data structures (something that at the very least a CS degree should help with)
-- Couldn't handle an intermediate SQL query fix involving a simple normalized schema
-- Couldn't tell you anything about internals or security, and
-- Didn't seem able to describe how they'd build a CRUD app.
All of these candidates had CS degrees from respectable-to-good universities.
Anecdotes are not data, but high-level data says the same thing. Our coding problem is very similar to the practice problem shown at [1] (though slightly harder). It doesn't require any tricky lateral thinking, deep language features, esoteric algorithms, dynamic programming, weird race conditions, or any of the other usual trivia BS - it's as straightforward a task as we could design.
And if you look at how much progress people make (see [2]), it's not much. The vast majority of people we interview have CS degrees, and two-thirds of them still get ~nowhere.
Meanwhile, the best interview we've done this month was with a guy who has been the sole developer for an ag business in rural Arkansas for the past decade. He has no degree and his previous two jobs were a sandwich shop and running a website for his church.
(Specific details slightly fudged for the sake of candidate anonymity, but not in a way that materially affects the point.)
[1] https://www.otherbranch.com/shared/practice-coding-problem
(Full-disclosure: mostly self-taught with several decades of experience.)
Just because it's a kind of visualisation of a distribution, the elite are not that far ahead of the competent, but there is a very long tail of shitters out there.
The guy single handedly created the best online chess site out there outperforming whole commercial teams in every area (performance, interface, features, whatever you choose). His work ethic is legendary and he pays himself below entry level salary (for US standards).
So 90% of CS graduates are useless, which it why, thank God, the inflationary pressure towards programmer jobs is far lower than the droves of new arrivals would suggest.
Therefore the top 10% are 100x by default compared to the middle because the middle is really a liberal arts graduate, not a computer programmer.
This just isn't true, I mean, my university has a graduation rate around ~65%
People who know the least amount universities talk the the most confidently about them.
Well, duh. Nobody is going to waste their time talking about something they know well. What would be the point? They already know about it. Anywhere you find people talking about a subject, you know they have limited understanding. That's why they are talking about it — seeking more information to expand their understanding.
Is x the average?
Is a programmer someone who just implements the requested features, or, like bluecalm commented below, someone like the Lichess creator who apparently does all of "performance, interface, features,", etc.?
What metric are we using here? Revenue?
Then again, this is nothing like the type of problems I work on a daily basis.
I don’t know if they’re looking for superheroes or people who can put in the work.
If the GP is reading this, I’d suggest increasing the timer a lot more (like one hour, what’s the big deal anyway?) and splitting the requirement and steps into smaller chunks. The “optimize for speed” remark in the instructions is also a bit confusing.
As for being stressed -- if this is too much, how will you do when asked to solve even harder problems with tight schedules and/or demanding customers?
Isn't the whole reason you are joining a team is so that you have a support network when such situations arise? The period of acting alone to join that team is a temporary aberration and is to be recognized as such.
You may as well be the business selling to those demanding customers if you have what it takes to handle going at it alone. It will be far more profitable. But if you don't have it...
And if I did have to do something like that at work, I promise my manager would give me more than 25 minutes to do it.
sum += (x > 0 && y > 0 && x < x_limit && y < y_limit) ? a[y][x] : 0;
>=-- joking -- :)
But your take on the stress component of an interview is way off base. There’s a huge difference in having stress on you to deal with other people’s problems vs. your own. An interview candidate may be in financial trouble and may have been struggling for months to find employment in a terrible job market. This is existential stress on a completely different level than a customer wanting a feature yesterday.
There's no such single "stress". It's all different. Being yelled at by mom is "stress", and so is being held at gunpoint. Not the same kind of stress.
Making the slightest mistake or wrong impression during an interview pretty much ends it right there and by extension kills your chances of employment. Your entire future career and financial stability for the next 5-10 years hangs in the balance at all times during an interview.
Not comparable in any way to "the customer is being annoying today".
> Very few people will finish the full problem. You do not need to complete every step to pass our interview.
A problem that an average candidate completes doesn't have a lot of differentiating power on the high end, whereas a problem where an OK candidate finishes 2-3 steps and a great one finishes 4-5 carries a lot more information.
Because like many people have stated, this is just a hand holding test. You're basically seeing if they can code. So you're testing how fast someone can take a list of 5 steps someone else gives you and translate it into code in under 30 minutes.
And I'll tell you, I've never, in my life, worked on code with a series of 5 requirements and submitted it for review after 30 minutes. That would be bonkers, because I'd probably spend at least an hour sussing out all the ambiguities and contradictions.
> I don't think how far a candidate gets on this particular test determines if they're OK or if they're great.
It doesn't with certainty, but measurement of this kind is always subject to some degree of error. The point of interviewing (especially at the early stage where this interview is applied) is signal, not certainty, and we do seem to be picking up on signal:
-- # of steps completed on the coding task is the strongest single signal of success at later final interview rounds. Candidates who have gotten offers (not from us, to be clear, so this is uncorrelated error) average ~1.1 more steps completed on our coding problem than candidates who don't. That's after we filtered a lot of the weaker results out, so it's sampling biased in slower candidates' favor. This is a larger gap between offers and non-offers than any of the other nineteen individual internal scores we record for each interview, iirc.
-- # of steps completed correlates with results on the rest of the interview in a way that is aligned with reasonable psychometric measures of quality, same as most of the other parts of the interview (see e.g. [1]).
If your point here is just that interviews involve somewhat artificial work and are prone to error - well, yes. Everyone involved in testing as a field already knows that and is working around it.
To use your concrete example, creating a problem involving "sussing out all the ambiguities and contradictions" for an hour would be asking a lot more work from candidates. If we did that, I bet we'd have people complaining about the lengthy workload of doing an interview (or simply not doing it, which would be fatal to us as a business). I would also worry that that's a much fuzzier skill to measure, particularly in a cross-organizational way. It's not that, in isolation, you could never create an interview to test ambiguity-resolution, it's that (at least in my judgment), it would be impractical for us to do so given what we are trying to do.
And finally, to this:
> I posit that a great one could finish 2-3 steps and an OK one could finish 4-5.
See [2].
-------
[1] https://bsky.app/profile/otherbranch.bsky.social/post/3lpcdl...
> I am not sure I'd get the adjacent mines part correct on the first attempt
Yeah, getting to that point was easy. Solving that problem is where I got stuck. I'm sure I'd get it eventually, but getting to that point and figuring that out in 25 minutes is asking a bit too much.
The floodfill part (the last one), on the other hand, is the one that stands out in difficulty compared to the previous parts.
I thought it'd be fun to take a stab at it in Python, which I haven't used in a while, but only barely still remembered that I could accept command-line input with the built in `input` function, something I don't think I ever used after writing my first lines of code 15 years ago. Then I figured I'd use just lists instead of a numpy array but had forgotten that [[0] * 4] * 4 would just create 4 references to the same list. And that pretty much derailed the whole thing for me, even though I was sure I'd get it done in 25 minutes or under :-)
Alas, it’ll be a few weeks and by that time I’ll have forgotten :)
Minesweeper?
Huh.
No wonder the PhD candidate who was supervising us was surprised I did that as a fun little side project in my first or second year at university.
Still, I'm not absolutely sure I'd have gotten to the end of step 5 (correctly working flood-fill) within 25 minutes despite having implemented the whole game in Java, REALBasic, and JavaScript over the years.
On the other hand, that's probably good. 25 minute coding challenge where you're not even expected to get to the end as per the instructions* is much better than "take this, do it over the next week, might take you 8 hours, we'll spend an hour talking about it" as I've been seeing.
* "Very few people will finish the full problem. You do not need to complete every step to pass our interview."
The thing which makes SEs good is being smart and having passion. Self-taught selects for that. For young people, take a smart young person and put them through a CS degree whilst working. They'll be instantly productive and with a little guidance can be successful. Over time they become exceptional.
That was my path.
On the other hand, most people I know have at least dabbled in learning to code. In this case the cost of quitting is very low and the cost of getting good is reasonably high, so it self-selects for people who excel at it and love it.
That said, if you only compare people with similar aptitudes, my experience is that people with formal training are better.
tl;dr: The most important thing to me is comparing past work, interview and personality. If you have a CS degree with no personal projects, you won't get a job offer. If you don't have a degree but have active projects with good code then I don't care if you have a degree. That said, I have been getting hundreds of applications for every entry level job I post so I can normally find someone who has both.
I love small problems like this. I couldn't get all the steps in 25 minutes, but I think I was not that bad. (I was in stage 4 before time ended, got distracted about taking input with Rust, heh)
I think, working with these small problems would give my ADHD brain a boost before I start my daily works. If more detailed problem, I would spend the whole day on it, smaller would not give me much push.
This was just right!
I do not disagree with you at heart, but I think there are so many pervasive issues with modern education that I am not certain a high level of intellectual aptitude is necessary anymore. Hell, I somehow have a computer science degree which I honestly do not deserve considering how little I understood (and still do not understand). Then again, I didn't go to some prestigious university for the astute either, so take it for what it is worth.
Nevertheless, I feel like I learned a lot of information and was exposed to many topics at a surface level of understanding. However, there is little I learned that I couldn't have learned by just reading the textbook. In fact, most of my assignments and tests came directly from textbook or were similar enough.
I was trying to brush up on data structures and algorithms last year. Not because I was looking for a job, but because I always wished I understood DS&A on a more fundamental/mathematical level. I like details and understanding the 'why' or 'how' more than the 'what.'
I remember watching lectures of a professor on Youtube (Dr. Skiena at Stony Brook University). All I could think was how there is likely a slim chance in Hell that I would have made it through his class nor Stony Brook.
Overall, I think most of the difference is down to desire and motivation. Many of my fellow students at the time, cared little about computer science or programming. Most did just enough to get the piece of paper so they could start making 'real money' as they often would say. If one is capable enough to teach themselves how to program, then I do not doubt those individuals are capable of learning computer science to a level equivalent to me. Sure, they might not obtain a PhD level of understanding, but then again, the vast majority of bachelor degrees do not either.
I still take on faith that a kernel is some black box, some thing that just … works. It seems to manage memory, threads to some degree, the file system…?
> I'm really struggling to believe that a majority of people can work through that material and then can't do the type of work that most developers do.
In practice, the way maths and sciences are taught to CS students is without demand for any ingenuity. Most of the time, students don’t prove theorems, they only perform rote calculations according to an algorithm. The few proofs that do show up are things like a few very basic proofs by induction, where students are basically given the algorithm how to solve such a problem beforehand. Statistics, same story – fit a problem to one of the template solutions you were given earlier. Physics, same story.
That’s what for me explains why in my experience maths graduates have been better programmers than CS graduates (as work colleagues). And why some people in this industry hold it as a badge of honour that they can’t invert a binary tree.
If there is a virtue to CS degrees, it's that most engineering tracks require an immense amount of coursework in hard sciences that require a high intellectual aptitude to complete.
I'm really struggling to believe that a majority of people can work through that material and then can't do the type of work that most developers do. And in the most difficult realms of development, I think you'd find a majority of the developers have advanced degrees in CS.
I think having a deep interest in software is more likely to traits you're describing (and described in the article). Some people who have that didn't study CS. A lot did.