Can you speak to that a little? I'm reading it as either programming challenges in a real environment, or analysis of previous code snippets submitted. In either case, I'm happy to see more companies doing this.

I'm always looking for ways to improve my interview skills, but I want to do it honestly. Studying to the test isn't fun for me, I would rather hack on something.

The trick is, to work in a recruiting context, you also want those tests to be standardized and repeatable. A lot of companies fall down on this. They have candidates do "real work", often in a pair-programming context. There a bunch of problems with this:

(1) "Real work" usually isn't standardizable, so you can't compare candidates

(2) Signal quality from the test is intensely dependent on who is doing the work with the candidate

(3) Two different candidates might end up getting "tests" that are wildly different in terms of predictive power

I have a bunch of ideas for pure software development work-sample tests, but I'm not ready to share them. The idea is simple, though:

* It's a realistic exercise that approximates actual day-to-day work as much as possible

* Every candidate gets the same exercises

* The exercises have objective (preferably gradable) outcomes

I want to keep my hands on a keyboard and I don't like airplanes (which rules out consulting) so I've applied at a lot of companies to be a ``security engineer''.

I can't tell you how many well-respected companies ask me to write min-heaps, depth-first searches, etc. I don't understand what they are asking me this for...It isn't even close to a realistic representation of what my day-to-day responsibilities would be. It is also an immediate turn-off....

Cramming these questions online does not work. I don't know what Facebook/Google et al are doing but I can spot a candidate who studied interview questions from a mile away. They may bang out a DFS by rote but that's just warmup; they should be able to talk about details, deal with changing requirements, discuss the algorithm/efficiency/design of their solution, be able to talk about underlying data structure primitives, etc.

To be fair the grandparent was talking about a specialized position in security engineering for which the generic coding interviews could be a bad match.

And I don't understand why someone who spends his time up to the eyes in crypto proofs, exploit code, and research articles should have to cram for an Algorithms 1 exam for each interview.

I've seen these questions used well (had a neat discussion with a start-up CEO in Boston once about reimplementing a toy Twitter in Scala), and really brilliantly (another start-up in Boston once asked me to describe my favorite algorithm and got a short, ad-hoc tutorial in lottery scheduling), and really, really badly (BigCos asking stuff you'd see on an undergrad-level exam from years and years ago).

Strangely enough, I'm praising the two startups that didn't offer me a job (though I'd really like to reapply to that second one), and condemning the BigCos (you know who I mean) that did offer me a job.

I'm not saying it isn't important to know, but I think there are better indicators for someone who will perform well on the job

It's no surprise these companies rejects many really good hackers (at least from people I know... yes, I admit the small sample size) and accepts these just optimize for algorithm preparations by doing a set of online coding problems.

I wouldn't ask a candidate to implement a min-heap (mostly because it's kind of obscure and I'd have to think about it) but a depth (or breadth) first search is something I'd expect any competent candidate to rattle off in a heartbeat, just because it's such a fundamental cs principle that gets used in lots of places and, unlike things like sorts, doesn't really have a good library to fall back on in practice.

In reality, I have not accepted an offer from any companies who ask me these questions because most of the positions are not exactly what I am looking for.

Also, I tend to disagree. Some of the best security guys out there never went to college. It's hard to expect these guys to know algorithms and datastructures even if they are really gifted exploit developers, or the like.

edit: To clarify. What this really shows is that I am not being interviewed by a security person...ie there is no special interview process for my position compared to a regular software engineer.

My cousin's employer has started using Javascript Under Pressure [1] as the first round of technical interviews - it lightens the mood, and fully half of the people applying for a front-end development position at his company can't finish the first one, which is literally to write a function that returns the argument multiplied by two.

[1]http://games.usvsth3m.com/javascript-under-pressure/

However, getting one of the tests wrong in 4th problem because of two chinese UTF-8 symbol having more length than "tiny" kind of irked me and reminded me why I do not do webdev.

I suppose this is actually a good kind of test, because real webdevs would actually have learned not to use s.length but use something else.

What is it though? This seems like a Javascript fail more than anything else.

Gah, that made me jump.

That would still be a notable effort, but I'd like to know who I'm hiring and for which qualifications (cat herding or design and implementation).

And would that matter?

When the teleprompter reading person applies for a security job they're supposed to do alone? yes.

Interview Process is: HR filter -> Manager interview -> Grandfather (Manager's Mgr) interview -> Peer meeting

The peer meeting isn't conducted in an interview setting. It's more informal, adapted to the needs of the applicant. It could be a lunch or a dinner and once it was even participating together at a hackathon.

We carefully select peers against "buddy bias" and encourage diversity in the workplace.

Our company is over a period of several years very successful in recruiting and have very little turnover. The biggest drawback could be that we are "too picky" and thus (at times) have problems growing fast enough.

The secondary reason to ask these easy basic dev questions is an attitude check. Someone who gets personally offended at being asked to do something that is easy for them has a risk factor for being a poor teammate.

I see statements like this thrown around a lot. Do you have precision/recall/F-measure numbers to back this up?

Depends on your language. Some are better able to abstract and express this `pattern' as a library.

We have a front-end and back-end test. I'm sure that at some point we'll grow large enough that solutions to our test will be posted online but I'm not going to worry about it right now.

"but only because a) he was really hungry at the time b) he really went out of his way to build up a lot of contacts with a lot of top-notch talent."

And that was really the essence of what I was going to say. I think after a while anyone who is busy and being paid in the manner that a recruiter (or real estate salesman) is is going to go for the easy route and the low hanging fruit. Especially as they get busier. Because back and forth is time consuming.

It's kind of like a satisficing model. You need to keep the client happy enough to keep sending you projects and no more than that. [1]

Consequently in theory a new and hungry recruiter is really the one that will give you the best results but that assumes they have a file full of candidates and that kind of contradicts being new.

[1] There is truth to that book that stated that real estate salespeople get more when they sell their own house than your house. 3% of $10,000 is not $10,000 and it pays to move on to the next deal in your book.

I've never seen a industry that needs a fresh, iron clad Application, but it has to be top notch. A hungry, smart developer could revolutionize the Realeste market with the right code, and eliminate Realtors to a history wiki. Plus--I would never need to look at their stupid smiling faces everywhere. And yes, I know a house is the biggest thing you will ever own--you need hand holding? That's all your full commission will get. Smart Realtors always cover their asses legally if you buy a Lemmon. This is just my rant on Realtor's--especially full commission brokers. The above post reminded me of how arcane buying a house is.

The large bodyshops, OTOH, seem like a colossal waste of time to me. I don't trust them, and the candidates they offer have been almost uniformly disappointing.

His "secret" is that process, which also allows him higher than average access to passive job seekers. Let's face it, many (not all, but many) of the "best" candidates already have a sweet gig and aren't pounding the pavement looking and mindlessly applying to every job posting. This recruiter calls them and they take the call, because he's not just matching asses and seats like it was a musical chairs problem.

Candidates like him (he placed me once 17 years ago) because he's not perceived to be a time-waster, and hiring managers like him because I get better candidates through him (albeit at a lower volume and I "can't afford" him for every role).

That's responsive to your first sentence; to your second sentence, I generally agree. I have the luxury of having a strong in-house recruiting team, who are willing to selectively and smartly use outside agencies when it makes sense. As a result, I get to direct the flow of nonsense from outside agencies at my in-house team. ;)

I treat recruiters like widening the funnel, and haven't found any correlation that suggests they produce better results.

I have found the best results is hitting people up through linkedin.

When was the last time you applied for a software developer job?

Who asks programming puzzles? Read interview postmortems. The answer appears to be "everyone". We are probably just talking past each other: asking someone how to sanitize an input or parse an HTML document during a face-to-face interview is a "programming puzzle".

Perhaps the real problem is that those companies have taken all the people who perform well at whiteboard interviews, and what's left are people who struggle.

Still, I wouldn't say the system is flawed because it selects a certain type of people. Clearly those people do well.

Having said that, if you talk to Google people who are close to hiring, especially about the process that connects "interviewers" to "recruiters", I don't think you're going to hear consistently good things about how the system works in practice.

http://qz.com/96206/google-admits-those-infamous-brainteaser...

Algorithmic testing has real practical implication if the role requires it, which a subset of the engineers at the aforementioned companies will use.

The other practical implication is standardization of the interview process, rooted around an academic background in computer science. It's easy to test whether someone knows a "fundamental" (whether it's actually fundamental to the job is another question) cs data structure or algorithm.

The idea is, interview for square pegs, since it is easier to measure the quality of that particular shape at scale. If there is a high-quality peg that happens not to be a square, it is a loss that the system is willing to accept, provided that a large enough % of the good candidates are squares.

Though I believe algorithmic knowledge is a lot more practical than people give it credit for. I'm not talking about crazy graph searches, or implementing A* in ASM.

Are you going to be efficiently searching strings, or reordering arrays, full time? Probably not, but you will likely come across a time where you will need to think about the consequences of your design, even in a simple crud web-app. And that stuff is important.

It could be also cargo cult.

If you are a startup, how you will add a technical founder or a first technical hire? Just by his/her ability to write O(log n) searchs in a whiteboard or you will dig deeper and make sure is the perfect match?

Early in the life of your company you're hiring for very long term, and you need generalists.

Unless your company is building a product around whiteboard coding or something.

"results in many false negatives..." is not some universal truth.

Is there evidence that shows current software interviews turn down way too many great candidates?

There should exist a large enough group of developers who have been rejected a number of times[0], who have gone on to be very successful[1] in their careers. Does this data exist?

I agree that the current process isn't perfect, and it does produce some false negatives and some false positives. The amount which, has yet to deter me from beliving that the system works, as intended. Certainly there are some fringe candidates, who are turned away, but I don't think it's as onerous as people are making it out to be.

Also, most companies didn't start out with unlimited piles of cash, nor could they have their pick of employees. They had to work at it, and I don't believe the culture of their hiring has changed significantly.

For the record, I struggle with traditional coding puzzle interviews. But I also know coding on the whiteboard is a skill, which can be practiced, and with time, improved upon, such that if you are a really good developer, but struggle with passing interviews, you can deal with that problem by becoming better at interviewing.

Or you know, create something awesome, and never have to worry about interviewing again.

[0] Failing one or two interviews isn't really indicative of anything. So this group would require consistently failing multiple standard coding interviews.

[1] How to measure success? I say gone to be a major contributor to the success of a company, or built their own company that is considered "successful."

It's an open question whether a negative for fit is a false negative or not. If someone wants to be a foo and the need that I have is for a bar, it's a true negative for us, but when that persons goes and finds success as a foo somewhere else, I'm not sure where to put them your data set.

Even for the false negative, how would I possibly know what career path they took after that? I certainly hope it turned out well for all of them, but I have no practical way, not to mention no incentive, to find out.

> Failing one or two interviews isn't really indicative of anything.

I'm asking for evidence that shows candidates who have routinely fail traditional software interviews, have gone on to be above_average -> exceptional in their careers.

My hypothesis is that if traditional software interviews have systemic and endemic flaws--that is interviews produce there are a lot of false negatives, there would exist a large group of people who have failed continuously, but have yet gone on to be good-to-great elsewhere.

Getting a no hire from one or two companies, would not classify a person as a "false negative", it could be (as others have mentioned) an issue with fit.

SQL Injection--not html parsing ;)

What does it mean? Throw engineering problems your CURRENT team has solved in the past at new candidates. I have seen only ONE startup ( even among the nastiest of hard problem seekers ) GET how substantial this is. So many of these companies are too busy to think about smart sourcing, at their own peril.

Test new engineers off past solved problems. End of story.

  we interview people whose only prior work experience is
  "Line of Business .NET Developer", and they end up
  showing us how to write exploits

Or: Zero prior software security experience. Zero prior Rails experience. Zero prior Ruby experience. Hired through a resume-blind work-sample process. Looks at Rails for the first time, 30 minutes later reports a vulnerability that results in a CVE and a Rails patch. Repeats. Now runs Rails internal review board.

I have other examples. Worth knowing: we aren't Rails neophytes (for instance, we're one of the firms that reported the YAML code execution bug) or for that matter crypto neophytes.

It's not about what technologies you use; I don't so much care whether someone writes .NET code. It's the "absolutely no prior work experience doing this and no flashy resume to get them in the door" that stick out to me.

It's just that Hacker News biased to a certain section of society.

At my last company, we did a traditional interview plus a pair programming interview. Eventually, we moved the pairing interview up in sequence, because it gave us far more information than the traditional stuff. It let us see what they could really do, and told us a lot about how they worked as part of a team. Mere questions don't get at that.

We also paired most of the time in the course of our work, which gave us a lot more room to take hiring risks. If somebody was bad, the amount of damage they could do was limited. If somebody was just uneven in skill profile, pairing let us take advantage of their strong points while mentoring them in their weak points.

I'd love to hear what else people using a Moneyball approach are doing to reduce risk and increase yield after hiring.

That sounds right to me, but I'm wondering if there's any evidence to support the claim.

Of course, this assumes that a trial (say, 30 days) can give a lot more information about the long-term potential of the candidate than a day-long interview, which feels like a reasonable assumption to me.

I'm not wholly disappointed, because 1) I finally got around to using vi, spending a solid week using it and only it, as my homework[1] for the company, and 2) you don't want to work someone that only hires clones of themselves.

[1] They didn't assign it to me, but I'm always looking for something interesting to do, and since I was serious about wanting to work for them I did it to ease my eventual on-boarding.

Be happy you don't work there.

The industry wants hordes of people who color by number with the newest frameworks, not hackers.

I didn't provide evidence, but did point out what I see as a smoking gun, which is the trend of temp-to-perm hiring.

Temp-to-perm is not conventional hiring; if you do it, you are not hiring the way AmaGooBookSoft hires. That you would do it at all suggests that the conventional hiring process isn't effective.

At the same time, temp-to-perm makes it harder to find people; it (almost) has to, since asking a candidate to accept a monthlong (or even weeklong) contract is onerous. People forget that candidates often interview at multiple companies; a week off work seems reasonable when you don't consider that it's a process that might repeat 2, 3, or 4 times for a given candidate.

There is clearly evidence that the entire hiring process leaves much to be desired. Temp-to-perm is obviously not ideal. I'm just wondering whether the problem is that companies are relying on the temp-to-perm pattern in lieu of better available alternatives, or if the temp-to-perm pattern is simply the best option because there is genuinely a short supply of experienced, easily recognizable talent.

(We will occasionally hire an ex-contractor, but that's the exception and they were genuinely a contactor first. No one comes through the front do applying for a full-time position and gets offered a contract-to-hire.)

Basically, in a 300 mile radius around Washington D.C. there is apparently a shit ton of little, bullshit consulting agencies. There are at least two in even the small towns. Bullshit consulting agencies are the type I'd expect to do contract-for-hire.

It's also, for no reason I understand, the only types of places that would ever look at my resume. Perhaps that is because the alternatives to the consultoware places are health insurance companies (which I patently refused to apply to) and financial services companies (also not terribly appealing). Those sorts of places always looked like they were exactly the same sort of poorly managed shit-show as the consultoware places, just that the client was in-house. Not much consumer-facing software being made out here, I guess.

Excellent information. You gave some thought to the process and stopped going for the legacy solution to the problem.

Would like to point out to anyone that you can actually do a version of this even when the company isn't doing the last 1/3 and being the one that is creative. That is you find a way to separate yourself from the crowd in a way that removes the resume, interview and credentials from the evaluation process.

My example of this is to go for the job that isn't being advertised or that the company isn't even hiring for so you aren't competing with the other person with a better resume.

I did this when after I sold my first company I landed a sales job at a tech company never having sold that type of product and really having very little experience (in that) at all. They weren't hiring so I wasn't being compared to anyone else just to whether I could do what I proposed to them. For that matter anytime I sold a product (for something for two different new companies I founded) at the start by cold calling I was doing a version of the same thing. They just evaluated whether they needed the product/service not whether I had something better than the other people who visited.

I guess the bottom line is whether you are looking or "looking for" you need to do outreach which is really just selling in a creative fashion.

When I got the first phone screen, I mentioned that your hiring page said to expect about a month-long hiring process. The response was that that was for someone with experience in the field. OK.

Since I had no relevant knowledge/experience, you (I'll use "you" to refer to Matasano) sent me WAHH, and said to read it, and contact you when I thought I was ready. I wrote back, asking how I'd know when I was ready (remember: no knowledge or experience). No response, ever.

I hope the prerequisite to interviewing at "entry level" in Matasano isn't full mastery of everything covered in WAHH. The last of my textbooks that I've looked at and thought "I know everything in here" was my middle-school algebra text (this one: http://www.amazon.com/Algebra-Structure-Method-Book-1/dp/039... ). And there could easily be something mentioned in there that I don't know; I made the determination by looking through the table of contents. Every one of my later textbooks mentions something I don't know in the table of contents.

I don't think I've just failed to accomplish anything at all since taking middle school algebra, and even restricting discussion to school math classes I have some accomplishments beyond that point. So the impression I got was of a ridiculously unachievable standard for interviewing, that maybe I'd be ready to interview after several years of work in the field. Or maybe not.

2. you sat around and waited for them to contact you. they have other, better shit to be doing, i assure you of this.

3. being aggressive and having initiative and believing in yourself is a big part of getting hired. if you don't have those things, you probably won't get hired in a competitive environment.

> What does "ready" mean?

That's hardly an irrelevant question; it's impossible to know that you're ready without knowing what being ready entails. Are you trying to select for people who are confident they can already do anything no matter what it is?

They gave you plenty of criteria; they gave you a book, and the understanding that they wanted you to be able to talk intelligently about the material that the book covers. If you're so incapable of self-introspection that you can't accurately judge how well you understand material without external validation, then you should probably work on developing some self-awareness before getting upset that a company doesn't respond to arbitrary requests for meaningless information.

I rephrased you because, if I just quote you, there's no indication that I understand what you mean by the words. You seem to indicate that I in fact didn't. I'm going to call that a success for rephrasing. If I get you wrong, I hope that you'll tell me what you actually meant, which you didn't do here.

Let me try saying things another way. If I ask, "what does 'ready' mean?" and you reply "it means 'ready to move forward in the process'" (my emphasis), you've defined "ready" by reference to itself, which is obviously incapable of being helpful. So I did my best to determine what you might have meant. The entirety of my complaint is that, lacking domain knowledge, and faced with a job ad saying "no knowledge is necessary to apply", I was told "you're not ready to apply, come back when that's no longer the case" and then, upon asking, was not given any guidance as to how to determine whether I was or wasn't ready. So, in answering the question "am I ready to move forward in the process?", all I can do is devolve it into the question "do I want this process to go forward regardless of the outcome (which I can't predict), or would I rather remain paused?". That's purely a question of, in the words I used before, how quickly I want to conclude the process. I can't refer to any other variables or goals, because I can't evaluate those.

> If you're so incapable of self-introspection that you can't accurately judge how well you understand material without external validation [...]

Nobody can do that. Try evaluating yourself without external validation sometime. If you score e.g. a "good", ask yourself "good in terms of what?" External validation is the only thing giving any meaning at all to evaluations.

This is not a hard concept here; I would rate it as slightly more complicated than walking and talking at the same time, and slightly less complicated than walking and texting at the same time. So, uh, you probably shouldn't walk and text because you'll hit a tree or fall into a pit. Just some free advice there.

If you're capable of forming memories, you're capable of self-evaluation without external validation, you goofball. "Good in terms of what"? Most people choose "how good [I was] before [I] started studying" and it seems to work pretty okay for them.

I kind of hoped you are at least looking on our answers or something :)

Given that background, given that people are trying to improve the recruiting of engineers, given the penalty applied to people who recruit for the standard package, given the rewards for those who recruit accurately to their needs, .... why does the "irrationality" persist?

If someone doesn't make the cut, what does Matasano say? Do you say what didn't work out, or is it the industry standard and horribly antiseptic "we have decided not to move forward at this time"? I'd understand the latter, but if there's a place breaking that mold, I think it might be you guys.

I'm assuming you are not interviewing every or even most.NET developers and trying your luck to find the one who can write that great exploit. So, from all the candidates with resumes that would be considered subpar before the process overhaul, how do you determine which ones have potential to write that great exploit?

We recently started putting sales candidates on the phone to cold call. The most critical thing we measure is how many minutes they spend on the phone. It's been a pretty strong signal of how they would perform if we were to hire them.

If you need 2 sales guys, hire 3, give them a month and fire the worst one.

That is horrifying to me as a developer, but apparently this is fairly standard for sales.

For an inside sales position, you're not all that far off though.

As I've seen it put in another discussion, "sales managers don't have any trouble firing people."

In other words, is the success of the sales people about spending more time on the phone in total? Or is it about spending more or less time per call?

Thanks in advance!

Our sales process is based on our reps cold calling businesses. We learned after a couple of mishirings that the most difficult part of the job is being able to manage emotion and taking rejection. For example, as a rep it is completely normal to be rejected in some manner 50 times a day. So if you get discouraged or upset after your third rejection, it is unlikely that you would have the motivation or drive to make more calls and therefore, you're likely not the right fit. All of this is usually summed up in the metric time spent talking across all calls.

While ultimately the time on the phone translates into revenue, we like to emphasize factors that is in our control. We can't control whether someone ultimately signs up or not but we do control how much break we take in between calls.

* Got into Ruby and Go (trying to get a feel for what language felt most comfortable for bit manip.)

* Got into assembly (for fun so far)

* Got a good appreciation for bit-level operations

* A bazillion more little facts and insights that help in daily work life

I bet if you polled how it's helped other people there'd be all kinds of great examples. For work, I'd say the best impact has been a personal drive to pentest our own product, resulting in a lot of bugs found and fixed. That included managing to root our servers via a product flaw, which was scary and amazingly fun.

So yeah, thumbs up, don't stop now (please).

As for the above catering to newcomers, sure but it ramps up incredibly and pushes some boundaries if you've never touched that stuff before.

Sorry, I'm a bit confused about this sentence. The roles your selling are the roles you're hiring for? I guess even if that's true I don't understand what you mean by selling roles before getting selective. Would love some clarification!

For instance on the challenges I liked that you put (largely paraphrased here) 'If you breeze through these we might want to talk to you.'. Having ploughed/struggledto set 5 now it's almost daunting to think people DO breeze through that. Granted, the stats you've published so far indicate those people are far and few between. It kind of puts you in your place, but the exciting part comes from knowing there's so damn much to learn still.

From my perspective, I'm probably not going to go out of my way to apply to a typical startup unless there is some reason to believe the renumeration is remarkable. However, I might be interested in applying to a company like yours as a challenge to myself.

If a candidate doesn't have "several hours" to engage in a process intended to lead to both sides investing thousands of hours over the coming several years, they're welcome to apply elsewhere.

For every time I've interviewed, I've done several hours of homework on the company, understanding their competitive position, reading their financials (if a public company), founder bios (if a startup), bios of the people I'll be meeting with, etc.

I'm not doing this out of desperation; I'm doing it because I'm picking the company just as much as the company is picking me. Conversely, I'm willing to "jump through hoops" provided I believe that the company believes that hoop is helpful in selecting strong employees, because ultimately, I want to work at a company with strong colleagues and the hoop is a signal that the company wants the same thing.

Fair disclaimer: I haven't applied anywhere in the last <long time> and in my 20s, I enjoyed the take-home programming challenges. To this day, I recall enjoying Acclaim (game company)'s take home test that was rudimentary AI for a turn-based very simple game. I crushed it, flew out to interview, and got an absolutely laughably low offer (par for the industry), but I still think fondly of "doing that homework".

I've also been asked head-scratchers before. I had a recruiter ask my GPA and SAT scores for a role in my late 20s. Wait, WTF did you just ask me?! I answered, and that firm (D. E. Shaw & Co.) was the most-talent dense firm over 5 employees that I've ever experienced. If I'd have indignantly replied that my SAT scores were irrelevant to the position and none of their damn business (both are arguably true), I'd have missed out on one of the best work experiences of my life (and two handfuls of those colleagues are still working with me today, 3 firms and 17 years later).

Anyway, sorry for the long response.

Shorter version: You're picking the company. Do your homework.

They're picking you. Do their (reasonable) homework if you believe they're assigning it for the purpose of giving you great future colleagues.

I'm currently employed, so I'm only spending 1-3 hours per week on my search.

I have enough experience, education, and honors in school that you shouldn't waste my time with pointless homework. I've never seen a job programming test that was a more accurate measure of ability than the homework and tests I took in school.

Look at it from the candidate's view. I send out a bunch of resumes. About 5-8 of them want me to take a technical test before they call me or before I meet them. 1-2 of them are willing to proceed directly to an interview without insulting me and wasting my time first. So, I focus my energy on the people who take me seriously and treat me like a professional.

It's physically impossible for me to do a technical test for everyone who demands it. I don't have the time.

If Google or Yahoo asked me for a several hour pre-interview screening test, I'd probably do it. If your no-name startup is wasting my time before an interview, I'll just look elsewhere.

Also, in my experience, a pre-interview screening test is anti-correlated with good work environments. Of course, they'll say I'm "not a team player" for refusing the test.

After I did a bunch of pre-interview screening tests, and only rarely progressed to the interview stage after doing the test, now I don't bother. I know my solution is correct, but still not even an interview after I do the test. For the handful that did interview me, I wasn't impressed by them. Why am I wasting my time with people who don't respect me? Why should I waste a couple of hours for the privilege of maybe getting a chance to talk with the hiring manager, who's already pre-qualified himself as a twit by making me take his stupid test?

I'm even reluctant for post-interview programming tests. Once, someone asked me to do an assignment after the interview, and implied I'd get an offer if I passed. I did it, I know it was a good solution, but no offer. There's even people who try to put free consulting in the test, like the guy who asked me to write a program that connected to Interactive Brokers and executed VWAP trades.

I'm even reluctant for on-site tests now. Just a couple weeks ago, someone asked me to do a test, said I could use whatever language I wanted, I picked C/C++, and then they laughed at me for using pointers. Why bother?

If you know your stuff, you should be able to evaluate someone by talking with them for 15-30 minutes. If you aren't willing to spend 15-30 minutes evaluating me, based on my experience and education, then I'm not wasting a couple hours on your stupid pre-interview test.

Yes, I may be missing out on good jobs, but they are missing out on a good candidate.

I think the question is: could they be used against you?

I don't see how they couldn't, and you wouldn't know.

You were voted up because it's a detailed and insightful comment. Thanks for sharing.