Use referrals.

Ultimately, if you cannot identify critical staff and don’t know anyone who can, you’re screwed anyway.

Exactly. Somebody running a company can't be an expert in everything. But they can be a good judge of people and have a wide enough network that they can get recommendations from people they trust.

A few of those trusted technologists will make a fine hiring/oversight committee. You won't need a ton of their time, so you can pay very generously for the insurance they provide.

Agree with this. Same applies to any expert: how do you identify an experienced marketer? Or accountant?

To hire someone with expertise to solve a problem in a domain where you aren't an expert: You find people who claim to understand the domain, and then discuss the problem with them in depth. Your goal is not to evaluate them in general but to understand how they analyze the problem and how they'd deal with it.

You may have to hire them as consultants. You have to dedicate a day or more to discussion with each of them -- maybe considerably more if your problem is complex. You have to reveal information about your plans and/or the condition of your systems that may make you anxious or embarrassed.

After you've gone over the problem separately with five or ten putative experts, you will have a pretty good idea which of them understand it (if any), and you will also know how to ask much better questions going forward.

If necessary repeat this process, using your better understanding to find better experts and gain a better map of the territory.

Once you can rank expertise in your domain reliably, you might hire one (or more) of the known experts, but the best ones will be hardest to hire. Or you may use your new meta-expertise in the domain to look for more junior hires who have an unusually good grasp of your issues. Also, you may be able to get the best of your experts to act as advisors.

This takes a lot of time and may cost some money depending on the domain. But not doing it will take more time and cost more money in the medium to long term (and even in the short term sometimes).

AKA become expert.

I wonder if the people who become fantastically successful simply have the ability to evaluate expertise, without having any of it. i.e. by "reading" people, in some sense.

This relies on the "expert"'s own evaluation of their expertise, which mightn't be accurate, because of Dunning–Kruger, or conversely, self-doubt. So, one would need to "read" those too.

You definitely can't trust people's self-evaluation either way. Lots of really skilled people are self-critical. "Reading people" of course helps but it is nowhere near enough.

But you don't have to become an expert. If I hire a plumber or carpenter, I cannot do what they can. But with enough work I can understand which plumbers or carpenters will do a good job.

... or climate change scientist? Or economist? Or president?

The problem of separating experts vs non-experts might be the single biggest problem society faces right now.

assuming you have a generally good education and brain to start with, a lot of hard work can usually accomplish it.

Think Elon Musk learning about rockets, back before he lost his mind.

I did it once for climate change, took a couple months, some math. Got there though.

Before he lost his mind? You mean the recent interview or before that?

This is the problem that credentials supposedly solve - degrees, certifications, professional body membership etc.

You don't realize the difficulty of the problem until you must hire someone outside your own areas of expertise.

Depends on the magnitude of the problem to be solved.

Small problem: Ask friends for someone they have worked with on a similar problem.

Large problem (building an org): Ask 5 VCs/CEOs/VIPs for intros to the best person they know in that role, under the explicit direction of “learning what good looks like.” Interview all 5 with open ended questions. Ask each for one for one more contact. Eventually you will learn to pattern match, and one of the folks you like will volunteer themselves or a friend, and you will know how to interview.

I remember somebody on this forum saying the best way to judge someone's technical prowess is to give a simple explanation of a complicated part of their field. If you can't summarize your knowledge you don't know what you are talking about.

Unfortunately, a non-expert can't tell the difference between a good simple explanation and glib, confident ignorance. So I think that's a good test for expertise, but it's not one that a non-expert can fully judge.

References and portfolio, at the end of the day.

The orange juice test is one person's answer:

https://www.intercom.com/blog/the-orange-juice-test/

Hire a recruiting company. There are recruiting companies that have a staff of full time consultants that will bring in a whole development team and manage the product.

At some point, you want to bring that knowledge in house, the recruiting company will then help you hire people or let you convert some of thier contractors (not thier full time people) to fulltime.

We have a lot of recruiters sending us "senior full stack engineers" that dont knoe the language they are most confident in and end up being dev-ops engineers who want to get into coding. A lot of recruiting firms dont know how to to get candidates either

I am not referring to recruiting companies that just send developers to a company. I am referring to recruiting companies that have full time staffed developers and project managers that manage the project for you in the beginning and then slowly help you build up your own staff.

Some will have a combination of onsite developers, “rural source” developers who are in a cheaper part of the US, contractors and outsourced developers but they manage the entire project.

I was offered such a position - the pay is above market but it required too much travel for me now.

Now how do you separate out a recruitment company who has solid experience engineers on board from one that doesn't but says they do? I've seen many of the latter and many founders who thought they were getting the former in the deal.

If you are starting a company, I would hope that you at least know what your requirements are and you could sit down with the project managers to hash out acceptance criteria and deliverables.

And then you find that the acceptance criteria aren't met, the deliverables don't work, and your ship date is hosed. What then?

How is that a different risk than every other project that ever existed - even those led by internal teams?

Preferably for an internal project you have a trusted delegation chain that would catch the issue early on.

Two scenarios:

Scenario #1 - you are a founder with a great idea, you hire a good software dev lead who then fills out the department. You work with the Dev lead to hash out your plan They organize it and after certain checkpoints, the dev team demos progress. As the founder, you are at the top of the chain making sure that there is a product market fit, and ensuring things are going as planned from a high level.

Scenario #2. You contract with a consulting company. Your project is led by a dev lead/project manager paid by the consulting company and they fill out the team using thier own staff.

They organize the team and after certain checkpoints, the dev team demos progress. As the founder, you are at the top of the chain making sure that there is a product market fit, and ensuring things are going as planned from a high level.

There is no difference in the delegation chain or the risks involved.

The main difference is that you are writing one check to the consulting company and they are paying the contributors and in the other case, you are paying employees individually.

The usual retort is that employees have more loyalty and ownership in the company. No they don’t. The employees at your company, if they are smart, have one eye toward thier next job just like the consultanting company has one eye toward thier next contract.

The goal of the recruiting company is to get candidates hired. That may or may not conflict with the ultimate goal of the client doing the hiring, which is to get the job done.

I have this problem internally at my company. Recruiting have crazy goals (hire 30 developers per month) so they push us to accept whatever they can get.

Sure, we can keep refusing the candidates but at some point you have to accept someone to get the job done. Even if you need to double check everything they do.

The reasons for that are too complex for a small post but believe me, there are many.

The post you are replying to said:

Hire a recruiting company. There are recruiting companies that have a staff of full time consultants that will bring in a whole development team and manage the product.

You are replying to a scenario that is just the opposite of my suggestion. The client company is not hiring developers at all. They are contracting out with the consulting company to manage the entire project. The founder is working with a project manager who is employed with the consulting company to develop a product. The recruiting company is doing all of the project management, development, QA, etc. The founder is hopefully coming up with acceptance criteria for the product since it was his idea.

After the project gets off the ground, then the recruiting/consulting company works with the client to develop in house expertise starting with finding a dev manager who hopefully understands the business and the technical and the dev manager works with recruiters to staff up.

You’re not hiring anyone at first. You’re basically outsourcing the entire development project to an outside agency. If you are a non technical founder, this may be the best way to go.

There is a local recruiting/consulting company that has been trying to get me to come work for them directly full time in the role of team lead, I just can’t do the travel right now.

I guess the assumption is that anyone or at least an inexperienced engineer would be able to point out an experienced engineer, while only experienced engineers are any good at picking less experienced engineers.

Deloitte has pretty good technical interviews. I'm sure they can provide you interviews as a service. Probably not going to be cheap, though.

Surely there is a big clue in how long they have been working for, how many projects they have done, what their roles were.

Based on my experience interviewing over the past several years, the companies that have a 'production' system seem to only value experience with their tech stack.

That’s because they have set something up and ironed out all the bugs so it will take some convincing to move them to a different stack.

There’s no easy way around it. In fact I’ve seen some teams too eager to rip up existing production workflows every time some new technology becomes popular and that’s not productive either.

I'm not saying that they should change their stack, I'm saying that they should consider that someone having extensive experience with related technologies probably knows the domain well and can come up to speed quickly on the team's shit.