1. Build outreach programs that are open-enrollment and broadly targeted (for instance, the crypto challenges aren't targeted at security industry events).
2. Explicitly overcommunicate the whole hiring process to candidates up front and sell the job, before attempting to screen. Actively try to correct for the adversarial nature of the process.
3. Drastically reduce the impact of phone screen interviews.
4. Administer standardized work-sample tests to all candidates regardless of previous experience; those tests have quantifiable outcomes (which we record).
5. Administer standardized on-site interviews with quantifiable outcomes (right now, they're "exercises", but their form might change over the long term).
The only point in our process where interviewers have a lot of flexibility in terms of what they do is the phone screens. We use the phone screens to generate predictions for how candidates will do on the rest of the interview, but we don't screen with them. (We would if the phone screen revealed a candidate to be a crazy person or a jerk, but I don't think that has ever happened).
So far, the things that have been most noticeably successful for us:
* Standardized, quantifiable results that allow us to generate a model of what a successful candidate actually does, rather than a gut feeling based on how well they answer questions, or how "confident" they are, or how "well spoken", or how impressive their resume.
* Outreach to the industry as a whole, not just to the small sectors of the industry we believed a-priori would generate the best candidates. Some of our best people have come essentially "out of nowhere", and recognizing that fact I'm especially interested in optimizing for that.
It is thrilling --- best word I have for it --- to find someone who's had no relevant job experience, hire them because they do unexpectedly well on a series of quantifiable metrics, and then see 6-9 months later that they are, say, showing us how to use BKZ lattice reduction and fourier transforms to exploit elliptic curve nonce biases.
As another example in this general line of thought, compare Grutter v. Bollinger and Gratz v Bollinger. Point-based racial discrimination is illegal, but "holistic" "individualized" processes where race is just a "plus factor" are legal. (Only discrimination in favor of non-Asian minorities, of course.)
I agree that this legal doctrine is insane.
Completely tangentially, since you mentioned relevant questions which are illegal to ask, I'd love to see an economic analysis of this topic. Specifically, the classical Akerlof lemon market paper assumes a complete inability to measure quality. But in the hiring market there are only a few dimensions for which one is unable to measure quality. That complicates the analysis significantly, so I'd love to gain some intuition on the topic.
In my company, we've hired everyone based on remote/part-time work, and paid tryouts. This seems to work well, but I don't feel like I can hire people who currently have full time jobs this way. I'm not sure how best to judge candidates who I can't observe working, besides a standard battery of technical interviewing and a lunch with the team.