Your experience comes from matasano, a company where people get paid to be hack, a dream job for many techies, with offices in New York, Chicago, and Silicon Valley.
Is it surprising that your experience is exactly the opposite? I would imagine there's a huge self-selection factor at play here with regards to who applies to work for each company.
(None of this is to say Steve's company isn't awesome - I'm sure it is - just that recruiting is going to be much more difficult).
p.s. - I agree with you that tech interviews are terrible - I just don't think anyone has got it figured out yet and what worked or didn't work at you for you at matasano is not guaranteed to work or not work for the rest of the industry.
If you read Thomas's recent post on Matasano's hiring process [1], they were looking to funnel people in. Steve's process seems pretty clearly designed to filter people out.
Compare the idea of sending would-be candidates books to study and the idea of an interview process that COMPLETELY EXPOSES candidates. One is trying to bring people in, the other is trying to drive them out.
Somehow tech hiring has often become an adversarial process. It shouldn't be shocking to us that approach makes it hard to find good people.
How can I know if an employer is worth 1-3 months of my free time until I meet my potential future coworkers? Demanding a candidate invest 1-3 months of free time before you meet them seems like an insult to me.
If your job uses a niche skill, why demand candidates be an expert already before you hire them?
I do have a problem with passing on candidates who were sold on our subspecialty, had an aptitude for it, but could not pass an interview on it "cold".
Are you hiring people who are brilliant, or people who are so desperate that they'll spend a couple months preparing for one interview?
Not your cup of tea? Totally fine. Not everyone is interested in doing security.
2. Package it, with all of the assets and utilities needed to get it running with "vagrant up".
3. Carve out some feature/features from the application, and replace them with stub functionality.
4. Deliver the vagrant app and a functional spec to candidates. Have them implement the missing feature.
5. Devise a scoring rubric (unit test coverage, lines of code, algorithms used, safe/unsafe APIs, performance, whatever). Mechanically evaluate candidate submissions.
6. (Optional) Devise a 15-20 minute on-site interview component to verify that the candidate actually did the work. We didn't bother with this, and multiplied the size of our team (NCC is the largest software security firm in North America) and had 100% retention. But it's a big concern for some people.
The reason is, of course, self-selection bias. On average, the people who applied for the tech college were just stronger students than the people who applied to the state college. This likely is because applicants to the tech college were more likely to be people whose primary goal was to peruse an academic discipline that interested them while the state school had a good number of students applying who wanted to party and attend football games. If you were to look at students who applied to BOTH schools, you would actually see a lower acceptance rate for the tech school than the state school.
I suspect that the same factor is at play here. I suspect that the average quality of a candidate who applies for a software security engineering position is much higher than the average quality of a candidate that applies for a enterprise software development position. The software security engineering position is an esoteric position that is more likely to attract applicants who are enthusiasts or at least very interested in the field. The enterprise software development position is more likely to attract anyone with a tech background looking to phone it in and collect a paycheck.
If this is true, while both Thomas and Steve have a difficult time hiring developers, they have a difficult time for fundamentally different reasons. Thomas is like the tech school - his difficulty is with getting candidates to apply. Steve is like the state school - his problem is with separating qualified candidates from "phone it in and collect a paycheck" candidates.
This is why I am skeptical of Thomas thinking he's got a much better way of hiring figured out. Maybe he does for companies that focus on security - judging by his Linkedin profile, he's only worked for companies focused on security. This is certainly valuable, but he has no idea what it is like for Steve.
By the way, I think many of us have seen this self-selection bias at play even within our own companies. Post a primarily Java job and a primarily Scala job and then try to tell me that the same principles apply for filling both positions.
Invariably, one of the top replies boils down to, "Hey! I'm a 20-something in the valley, and I see it completely different!".
Well, yeah.
"After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried."
I'm not sure if what you're saying isn't true, or if I'm just biased and not able to see Atlanta from an outsider's perspective.
I left because I wanted to live in a big city, not in suburbia. Atlanta has some of the plusses of a city, but they're relatively limited compared to LA, SF, NY, DC. It's more a collection of suburbs than a city.
If suburban life is what you want, Atlanta is fine for that. Honestly, most of Silicon Valley is suburbs, outside of SF itself. And they aren't as appealing as Atlanta suburbs, with many more trees and a lot more space. If that's what someone wants, Atlanta is a better choice than pretty much anywhere in California.
One of the best things about California, particularly in LA and the Bay Area, is that there are more people living their lives according to their own unique desires than anywhere else I've ever been. Having "odd" choices about your lifestyle or having crazy dreams and aspirations is the norm here, and that's something I want around me. It's a plus for me to be surrounded by that kind of thing. There's some of that in Atlanta, but it's more localized to certain areas. Here it's everywhere, just how things are.
But I do get your point about "odd" lifestyles. We're certainly getting weirder here, but I don't think it can yet compare to what you see in NYC or the Bay Area. Still, there's a lot to like here, and cost of living alone is a great argument for Atlanta.
Atlanta is by far the technology hub of the southeastern United States. It has no competition to the west until you reach Houston (and even that is mostly specific to the energy industry). It faces only mild competition from the Research Triangle region of North Carolina, and beyond that it's a technology DMZ all the way to Chicago or the northeast.
It's home to Georgia Tech, perennially one of the top 5 engineering schools in the nation, and pumps out thousands of new programmers per year in addition to the thousands who migrate here. It has a highly diverse business base, not tied to any one predominant industry, and its Buckhead and Midtown districts have one of the most thriving startup scenes that you'll find outside of SV or NY.
Of course, if a Georgia Tech grad is in the uppermost percentiles, then odds are he or she would be enticed by Google, or a west coast startup. I'm certainly not saying that Atlanta is in the same league as San Francisco, or even New York (at least not its finance/quant opportunities). But my earlier point was that these environments are extreme outliers, and virtually no other city is comparable to that either.
My experience: exactly the opposite.