There are people who can talk through challenging problems at their former companies and how the problems were solved. They can tell you everything you'd want to hear because it's true. Except…they didn't implement it. Maybe they are best friends with the person who did and understand in detail about the tradeoffs and the neat hacks and the insights learned along the way, but couldn't build it themselves.

Those are the "smooth talkers" of the engineering world. Those are the people you can't catch just through a verbal interview.

On a related note:

> If you can’t tell the difference between a smooth talker and strong technical competence you are probably interested in the wrong qualities. I have interviewed enough now to see why some companies cannot figure it out. Ask yourself if you really actually want a senior or a strong junior.

My dude.

Look at who you're replying to.

I agree with this. I was a hiring manager, and there are those that can really talk technical, in detail. You really think they know what they are doing, how to solve complex problems, how to come up with solutions. You put a keyboard in front of them (or pencil and paper), and they go "uhh, errr, ummm." and fail miserably.

I think until you have interviewed a LOT of people, it can be hard to quickly spot this. Some people are masters at telling you how someone else solved the problem as if they solved it, but they can not solve it themselves.

you can have someone who is a whiz at practical and specific solutions, who thinks critically and analytically and just gets an enormous amount done WELL. And empowers those around them to boot!

they have the reverse problem to speaking about other peolle’s work as their own. instead, they speak of their own work as teamwork.

this effects many great people. also women and poc are particularly likely to do this because they have been socialized to not speak too highly of themselves. “model minority” etc.

if you as an interviewer are already skeptical of what someone says, you will increase false negatives with people who you are asking to verbally “prove” their work and yet have cultural memories of being penalized for “bragging”. they’ll describe a solution and downplay it as challenging or hard because women aren’t likes le when they’re the smartest person in the room, etc.

an interview process should seek to understand many skills: practical, implementation, execution, problem solving, design, high level, communication skills.

a varied process that focuses on a few specific skills, one at a time, is likely to convey the most accurate signal.

False negatives are expected, and honestly probably good overall and in aggregate, because it decreases the odds of a false positive. One of my first bosses that involved me in the hiring process told me one day that the point of interviewing is not to find reasons to say yes, it's to find a reason to say no.

I've seen people that were entirely qualified for the position be rejected at the company I work for because they made some totally understandable mistake - I'm talking about people that took the time off to take a 5-hour on-site coding assignment, and made a mistake but would have had a passing (and possibly good) grade on an academic evaluation.

And now we have 5 open positions and nobody hired for them.

What is your threshold for "a lot"? I have given a few dozen interviews. I always ask at least a couple background questions. The number who even have a polished delivery for that part at all are a minority, and the couple that tried to bullshit me we're painfully transparent. Maybe I just haven't done enough yet.

If you feel that they're talking about a problem someone else solved, ask them that directly. (Did you work with others? etc) If they're lacking on the technical details either it's been a long time ago or they didn't do it.

I would think they and their colleagues discussed alternatives together, collaboratively in for example a Slack chat — so an interviewee can give you good replies about the thought process and alternative solutions that were considered and discarded. I would assume. Or maybe the interviewee him/herself came up with ideas, that his/her colleagues realized weren't going to work, and explained why, for him/her. Then s/he might be really good at describing the thought process.

Basically, they volunteer technical challenges they're aware of while simultaneously telling you what the high level solution is. But then you put a terminal in front of them and ask them to set up Postgres in a star schema with some dummy data, and then to write a query joining the two tables they were talking about before. Despite Postgres being on their resume, they'll completely flounder and not even know they need semicolons to terminate commands. Their joins won't just be wildly inefficient, they'll be syntactically incorrect and refuse to run. They won't be able to create, insert, select, truncate, drop, etc. They don't know how to create an index and can't mention any of the options for indexing, let alone the default provided by Postgres.

Keep in mind this example is just meant to be illustrative. Thinking through how to fix the scenario might not generalize to all the ways this can manifest. The kernel of how this arises is a person like so:

1. They read a lot about technical solutions at a high level. They can follow that if you have problem A then you need, roughly, solution B.

2. They have no contextual flexibility or practical foundation for understanding their solutions. They might have read Designing Data Intensive Applications, but they can't actually code and have never administered a database. To the extent they understood the book, they only internalized low hanging fruit.

3. They are charismatic, or ar least comfortable talking about technical topics. They will try to lead the conversation as much as possible, which is where you see them volunteering technical challenges and then offering solutions. But if you force them to answer heavy technical questions which drill deep into a specific area, they'll probably try to zoom back out.

edit: yeah upon re-reading you’re talking about completely obvious lack of practical experience... fair point

Even though the whole thing could be very simple. Another thing is, they will come up with reasons that the issues with the user story/task for code/solution is due to environment or some other reasons like tools, framework, scalability and "bs".

That's what that asshole is looking for. :)

Though to be fair, if you come up with that strategy and can't do -anything- at a sql console, I'm going to ask how you normally interface with the database, because that's like a Linux expert not knowing how to use tar or ls or something.

As for the Linux/tar piece, I've used Linux on desktop for a few years(both Ubuntu & Fedora) & have used Suse and CentOS for servers for much longer.

I can tell you tar means tape archive. I can tell you I mostly use it with gunzip to compress it. I still google/reverse terminal search what flags to use with it both when archiving it and unarchiving it. I could probably remember some flags if I spent enough time thinking - why would my brain waste that much effort though?

I don't work as a backup administrator. I have better things to worry about knowing/having present at the forefront of my (admittedly human sized) memory.

Do you suppose you could have made this point without calling me a name?

If you can explain how to design a system and you can do it, but don't know the exact commands off the top of your head, my comment isn't describing you. I don't expect people to e.g. know awk like the back of their hand, or to write perfectly compiling code on their first try.

But even if you don't have perfect recall of the commands, it should be pretty clear whether or not you've ever opened an editor and done basic implementation. If the GUI is your thing that's fine. But your knowledge must have some practical foundation which demonstrates you can actually walk the walk.

Imagine you're interviewing a candidate and they're talking through how to design an analytics service. They begin talking about e.g. database architecture, and how this type of data is most appropriate for a star schema. They start talking about the tradeoffs of row versus column orientation. They mention they'll need to do indexing for performance and talk about the index space versus query speed tradeoff. They say they'll do joins on the x and y tables.

Is this demonstrating deep understanding though?

Basically it's like someone else said. They read a book and know a lot of answers, but they can't do the most basic implementation of a solution.

No, it seems about on about the same level as being able to paraphrase the abstract of a paper about the system. I would not take it as showing that someone has read past the first page. A high-level overview just isn't enough for that. You have to ask your own probing questions too. Limiting the conversation to the particular problems they bring up is essentially taking them at their word when they claim to be skilled. I've seen lots of occasions where trying to drill down for a bit more detail on some part of what they talked about consistently came up empty (without going anywhere near sitting down at a computer to write a fizzbuzz equivalent).

Look at it this way. I was able to understand all of the maths proofs taught at my degree, but I could not have come up with them myself.

Imagine putting a math problem in front of someone and asking them to solve it. They correctly identify it as a system of linear equations. They volunteer that they would solve it using x algorithm which has a time complexity of y.

Then you ask them to actually solve it, and they can't even make the first movement towards doing so. They mentioned LU decomposition, but they can't even do Gaussian elimination on paper. They don't know what elementary row operations are. They can't obtain an augmented matrix or put it into (reduced) row echelon form. They don't know anything about linear independence or the rank of a matrix. You put an inconsistent system in front of them and they keep banging away at it, determined to find a solution...etc.

That's what it's like interviewing one of these senior engineers. It's surreal - they confidently pattern match the problem using limited heuristics, and they toss away low hanging fruit to demonstrate knowledge. But when you ask them to do something practical and specific, they either refuse and zoom out into abstract-land again, or they hopelessly fail.

What is system engineering?

It's not just you; the entire IT industry is suffering from systemic curriculum vitae bloat. That's why the working conditions are so bad in the professional sense.

Spoiler: "engineer" as a title for someone who does computer programming and software development without a license is perfectly fine and acceptable in the USA. In Canada, however, it's probably not.

For the record, the National Counsel of Examiners for Engineering and Surveying, which is the US body that regulates engineering licensure and "Professional Engineering", recognizes Software Engineering as a branch of the engineering disciplines:

https://ncees.org/engineering/pe/software/

"Professional (Licensed) Engineer" is not typically used in software development, but if you care about having this credential, you can become a credentialed Professional Engineer in Software Engineering.

Google defines engineering as "the branch of science and technology concerned with the design, building, and use of engines, machines, and structures". An engineer is "a person who designs, builds, or maintains engines, machines, or public works". Software engineers certainly fit that definition from my perspective, because software is a type of virtual or abstract machine.

Software engineering certainly qualitatively feels like other branches of engineering to practice, such as electrical engineering or computer (hardware) engineering. I've never designed a structure, but I imagine the principles are the same: understand the requirements of the customer or sponsor; devise a design that accomplishes those goals within the constraints of the medium you're working with, using principles of scientific reasoning to evaluate what is possible and whether a design will meet your needs.

Last but not least, software can be just as critical to human life and safety as the artifacts designed in other kinds of engineering. (But being critical to health or safety is not a prerequisite for something to be engineering. It's still engineering if you're building a rocket that only robots will fly on, after all)

There may be people whose work on computers does not constitute software engineering. Running some calculations in Excel is probably not engineering. But I think there are plenty of us who build large scale systems that need to operate with high availability under demanding requirements, the properties of which need to be painstakingly planned and analyzed and sometimes mathematically proven -- we who are trained as scientists and engineers, and who apply these principles in our work -- many of us consider our work to be engineering, and consider ourselves engineers. I certainly do.

Simply knocking in a program as a code monkey or hacking haphazardly on a program until it works in some way which isn't formally defined isn't an application of scientific theories in computer science into a practical product, which is what defines engineering.

You have a very narrow definition of the term.

> "Principal engineer"

"Look at who you're replying to." meant "tptacek" or "tyre"? Or both - a general advice?

A certain level of quality is expected, not labeling one selfs as engineers after 1 month bootcamp.

https://www.incose.org/systems-engineering

Good point. In many countries getting a degree in engineering takes way, way more effort than in computer science.

Then it takes 10-15 years of work to be called "senior engineer".

Because of this their hiring process was aimed to hire more "strong juniors" even tho they didn't realize it. A strong senior is a person who completely changes how you are even approaching the problem or someone who shows you problems you hadn't seen before. They often see tech in the context of business as well.

As you said, usually, they can't be bothered about learning the new framework of the month or new backend language of the month, but, they have a set of battle proven tools to solve tech problems for businesses.

P.S: I do find it hard to distinguish between bullshit talkers and people who actually get stuff done in the interview process.

Worked at a company which did nightly data imports. Things worked until 'companyx' became a cEient, and the imports were huge. They would take 18-20 hours. Then longer. eventually they were touching the 24 hour mark - unacceptable. Client's data would be more than a day behind. Granted it was a moderate amount of data, but shouldn't take that long.

I was 'new' there - only started a month before - and the rest of the team who'd put this together had been there a year or more. I reviewed what was in place, took a couple of days, and got it down to an hour. Then worked with the existing team and we got it down to under 30 minutes with some tweaking.

I do see some eyes rolling when I tell that, as I know it can sound terribly self-aggrandizing. However, I had a decade of experience at this point, and the rest of the team was just out of college; they'd never faced this problem before. I basically just took the data and imported in small chunks in to in-memory tables (to avoid hitting the disk), and copied those to disk every X rows, and dropped indexes until everything was done. It wasn't rocket science, but did take someone who had a deeper understanding of DB mechanics.

As I'm telling this, I always realize they have no way of verifying this, and essentially I'm just another bullshitter. The more believable I sound, there's an equally high chance I'm either really good, or just a really good bullshitter, and nearly every time, the person I'm talking to has no idea how to tell the difference. It's worse as you get older, because the younger folks just think you're waxing nostaligic about the 'good old days'.

You can't just say "I did X in Y by using Z": you need to begin by explaining that once upon a time there used to be a thing called Y, and on that thing it used to be very hard to achieve X, but in those ancient days there was also a tool called Z, etc.

CBI is a fairly effective technique for general interviewing, because you uncover how people actually behave rather than how they like to think they behave. Most of the gold is in the follow up digging questions, which should separate the bullshit answer from a real one.

I think this is overstated. Disruption for the sake of it is often not that helpful in the context of the business (although, in fairness, you do go on to state they often see tech in that context).

I much prefer people who are delivery focused to those who are overly idealistic or want to change everything out of the gate: a good senior understands priorities and knows when to make a trade-off to live with a sub-optimal situation or solution in one area in order to deliver greater value elsewhere.

Do you have problems to solve or not? Often the problem isn't really a problem, except that your current 'solution' is making it so.

I've untied a lot of gordian knots in my career. It really is a thing.

And if you don't have big problems to solve, why do you want a senior developer then? Just hire a junior and keep on going as usual.

Delivery focus is good, till you realise that it's often just delivering status quo for years on end.

That said, nobody is talking about disruption, just wisdom.

It just so happens, sometimes that wisdom will tell you that shipping shit out the door in the name of delivery focus is going to cost you more than its worth in the long run. Calling them overly idealistic to justify your laziness just makes you look bad.

I'd say a truly good senior can tell the difference between a startup and a mature company and adapt accordingly. It takes one set of skills to ship shit out the door as quick as possible and an entirely different set of skills to come in and clean up the mess the kids left behind.

This is why you are selling. Not buying. Because nobody wants to clean up your delivery focused mess.

Because hiring a senior to ignore the problem is a good way to waste a lot of cash.

I also don't appreciate being called lazy, and especially by somebody who knows nothing about my business or my team.

Please try to keep your tone civil in future.

I didn't call you lazy, I said that shipping in the name of delivery focus was lazy, or at least your argument about idealism was.

Fact is, shipping quality is the far more optimal solution and always will be. Making the trade off and adding technical debt is never a worthy trade long term. The only people who gain from it are you and your team. The rest of the company eventually grinds to a halt and begins taking more and more shortcuts around the code which just reinforces everything in a viscous cycle.

You might find this useful: https://youtu.be/DngAZyWMGR0

I was talking about prioritisation. I am very wary (or perhaps jaded) with people who want to fix everything all at once with no regard to the wider effects on the business of doing so.

I didn't mention anything about quality with respect to what we do choose to deliver, although I can assure you that user experience - of which quality is a key facet - is our utmost concern.

I thought my intent was clear, but sorry if not, and hence my comment about overreading. I wrote very little from which you (and you're not the only one) appear to have extrapolated quite a long way.

That analogy highlights what’s irritating me about this post and this entire thread discussion - there are 1,696 players in the NFL, each with an average salary of $1.9 million. When people talk about hiring “senior engineers”, they behave as thought they’re auditioning an NFL quarterback - the 1 out of 100,000,000 people who can actually perform at that level. For a salary of, on average, about $100,000. When you start out - offering comparatively little but looking for Tom Brady, you’ll pass over pretty much everybody, because the person you’re looking for won’t just not interview with you, they probably don’t exist. After a few months, you’ll relax your standards, and after a few more, you’ll relax them even more and end up hiring a comparative retard like myself - somebody with only a mere 25 years of hands-on development experience and a bachelor’s and master’s degree in CS along with a couple of publications. But if I presented in the first few rounds of interviews, back when you were looking for the guy who could derive the tortoise and the hare algorithm in 30 seconds in front of five people in a boardroom, you would have passed.

To be fair it is environment dependent more than anything else. Forget competence, charisma, intelligence, and everything else about the candidate that could bias their selection and instead look at processes and code already in place the new candidate is jumping into. Does the environment strongly favor original ideas/solutions or does it dictate the narrow acceptance in the most narrow of boundaries?

I have been on both ends of this as well. It is common in software for shops to define success in extraordinarily shallow terms such as whether you are using spaces versus tabs and indenting properly. Another example I experienced last year is whether you should write a giant monster configuration or a small function that receives a single argument. The reasons why shops coat themselves in blankets of code style and configuration is because they typically don't trust their developers and instead strive for a normalized baseline. They are looking for wonderful solutions, but rather task completion.

The lack of conformance isn't necessarily an indication of lower capability, but it is an indication of incompatibility. Competence and conformance are wildly out of sync when the candidate is misjudged relative to the work available. That is completely an assignment failure opposed to a candidate failure. Having gone through this myself it has taught me to ask very probing questions, as a candidate, during the interview. If, as a senior, I can determine I will not be a good fit I will happily disqualify myself.

> A coach like Bill Belichick might have a fantastic, detailed answer demonstrating a thorough understanding of every aspect of football past and present, but he could never make the throw himself.

Would you really hire a coach to be your quarterback? Is that a thought you would really entertain? Even if that coach could do that job he/she would be more valuable doing other work. I would consider this a solid example of interviewer/assignment failure.

Like what questions? I'm assuming the work you're looking for is more along the lines of "wonderful solutions" as opposed to "task completion" - is it as simple as asking, "How anal are you guys about code syntax and whatnot?" and, "How hard are your problems actually?"

Or is it more subtle than that?

What if I were to provide a solution that executes much faster, requires less documentation, passes test automation, and is a quarter of the code but ignores the framework or standard code style?

The standard DOM methods perform thousands of times faster than other options for interacting with markup. I can prove this with numbers. Code like that is not popular. Will I be allowed to write unpopular objectively superior code?

If I can reduce the application build from 5 minutes to 5 seconds will you let me rewrite the build from Java to Typescript?

A/B testing is a powerful way to determine preferential user behavior and a measured increase in conversion. Will I be allowed to write inward facing experiments to test developer behavior?

What if I provide a function as a solution the makes use of scope and nested functions but offers no support for inheritance?

Is it better to complete a task in 1 hour with original code plus tests or extend existing components with risk of regression and 4 days of effort?

There's a great quote from Lou Montulli[1]:

> I laughed heartily as I got questions from one of my former employees about FTP code the he was rewriting. It had taken 3 years of tuning to get code that could read the 60 different types of FTP servers, those 5000 lines of code may have looked ugly, but at least they worked.

Those frameworks and existing components most likely have a lot of hard-won experience embedded in them, and I would be uncomfortable hiring someone who did not appear to understand or appreciate that.

See also: Chesterton's fence[2].

[1]: https://www.joelonsoftware.com/2000/05/26/20000526/

[2]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

https://en.wikipedia.org/wiki/Invented_here

And I was not being condescending BTW.

> The greatest movie critics probably can't direct or write for crap

They may know a lot about how a film is made. It does not follow that they could sit in the director's chair, or write a decent screenplay. (And yet, they could still be great at critiquing films!)

This is a valid point. Just knowing how something is made doesn't necessarily mean you know how to make it. It's the difference between a designer and an implementer. Or architect and builder.

That last one is a real problem-solving strategy, I can't tell you exactly how to balance a Red-Black tree but I know what a Red-Black tree is and where to look up the algorithm for balancing it and I think that's good enough.

One of the master skills for interviewing is "don't choke." Often good people will make mistakes under the pressure of interviewing and will drop out.

When it comes to a practical session there is the same issue that some good people will choke. Just the sample of who chokes in which environment when is different.

Issues like this turn up in all cases where people try to measure merit. For instance, it is known that some low-SES (socioeconomic status) people will choke on standardized tests like the SAT or IQ tests. It is also known that some high-SES people are as dumb as posts and will have that revealed by standardized tests which are harder to bribe or bullshit.

The hostility towards testing is explained by the combination of those two groups: primarily the Emperor doesn't want you to see they have no clothes, but people have sympathy for poor people.

Exactly this.

Too many companies are availability focused instead of ability focused.

The reason? When your processes and engineering are weak, you need people available 24/7 to put out fires.

When I interview for a position, I'm interviewing them as much as they are me.

One of the biggest asymmetries in the whole hiring process, is that of course every company will tell you they have the best development processes, frameworks and code to work on. They may even believe it because they don't know better.

Then you take the job and are stuck fighting fires on a big ball of mud codebase that takes over your life.

If you want to standout as a candidate, try and probe and really find out how good their 'culture is.' Interview them back.

One of the best questions for getting to the real answer is: "What are your expectations for availability?" If they expect availability from you after hours, then their stack is likely unstable because the need availability to keep it running.

If you are lucky enough when you are senior and good at what you do, you don't have to put up with that.

I often tell them about a the third of the way through the interview: "I'm not an availability guy, I'm an ability guy." They often are surprised by the statement. And the discussion that follows it usually tells me whether I want to work for them or not.

What? It requires a special craft to design simpler components. Most of the people don't even see it. This is where your senior skills will shine when you'll make it work like a charm as compared to the previous state.

If you don't see the friction or can't reduce it then please take the first exit out.

The primary difference between simple and easy are decisions. A good senior will spend more energy on considerations for appropriate decisions than the actual work.

This is surprisingly often not understood, even by people I showed the video. And I am not sure why. But I do think it's necessary in out field to start understanding this much more deeply, especially for senior engineers.

For e.g. - for production deployment, I could either put the entire office upside down. Fires across the departments. Broken applications. Rollbacks. Or, I can automate all of our QA test cases and have a one-click deployment. No huss or fuss. I'm sure after having this experience, someone will definitely say - well, this was easy.

Anyway, good discussion.

I like to state it as complex vs. difficult.

Finding a bug in ten thousand lines of crappy VB code code is 'difficult', but not fun.

Writing an space/time efficient implementation of level set topology optimization is complex.

Senior developers usually love complex problems, but hate merely difficult problems.

What? I feel like you have a very narrow-minded idea of what sr engineers want.

Laziness: The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful and document what you wrote so you don't have to answer so many questions about it.

Impatience: The anger you feel when the computer is being lazy. This makes you write programs that don't just react to your needs, but actually anticipate them. Or at least pretend to.

Hubris: The quality that makes you write (and maintain) programs that other people won't want to say bad things about.

(That quoite's originally from Programming Perl 1st edition in 1991, the explanations I think didn't show up until edition 2 in '96 or so...)

Most of interviews I've done to candidates ended up in discussing technology trends and googling around for cool open source projects, libraries and so. This, to me, is a good indicator - as long as you bring them up when discussing relevant problems, this means you thought about a problem and researched prior art to avoid re-inventing the wheel.

Also we tend to end up discussing pro and cons of any given technology and so on. This at the end is a key indicator you're talking to a passionate developer. And passion usually makes someone good at programming.

When it comes to soft skills, usually having such a kind of discussion you can figure out also someones behaviour in most work scenarios (to me, having a good "discussion" mode means you're likely to be fit for team work).