I got comments once from judges in a startup competition and they were harsh, and offered no imaginative answers.
In most selections that draws a "pass or fail" line among applicants, giving feedback benefits neither. I explained the reason in my comment in this tree ( http://news.ycombinator.com/item?id=68895 ). I assume whether you're called back for YC interview or not is this kind.
What I didn't explain is that there are other cases; where "the few stand-outs" compete and ultimately the winner is chosen. If you couldn't win because of your shortcomings (that is, you would've won if you hadn't had them), then you should know what dragged you. But you don't need to worry; in such cases, somebody will tell you the reason you couldn't win. Most likely, in such circumstances, the officials make comments on each candidates. Even if there's no official comments, sombody inside will eventually tell you that you were so close.
Of course anybody who wants to improve himself needs feedback. But when you're in the first kind of selection and can't make it, those who made selection are wrong person to ask feedback.
Also, it would be great to see a YC applicant actually develop a system for the applications themselves making feedback and follow-up easier for YC staff.
So, if systems were in place to handle these types of apply-check-judge-accept/reject-feedback-followup processes better, I'm confident better outcomes could be reached.
Anaphoric's comment (http://news.ycombinator.com/item?id=68979) such a system could be used by that manager who rejects at random.
The same remark can be applied to other fields; recruitment, record companies, auditions to cast a role, men/women relationships, .... what's common here is that, although you can tell success or failure retrospectively, you cannot nail down the systematic criteria beforehand.
Not to mention the issues that come up once the system is changed. If the system is changed next year, and some companies that failed under the old system would pass under the new system, wouldn't that be a huge problem?
You're taking people at the top of their game, with knowledge in the industry and technology, and having them make a fine-tuned judgment call.All this for a dynamically changing business environment. If the decision is of a well-considered and thoughtful nature at all, that's going to be very hard to systematize. The more you formalize it the more you lose your special sauce. There are qualitative scorecard-type solutions, but those by nature are personal opinions and would have to remain proprietary.
I exaggerate, of course, but whatever reason you get I can guarantee won't be satisfying. I mean, you can go to the store tomorrow and pick up a huge stack of books on selling, which all boil down to "Sometimes you strike out for no reason at all. Keep trying" In a vain attempt to pump readers for money, some books give you a huge list of reasons for why you can strike out. I'm a firm believer in controlling all those factors you can -- wear the power tie, turn the temperature down in the office, sit in the proper locations in an important meeting, use body language and NLP, show instead of tell, etc. But I've never had a client say "You know Daniel, we really like your work, but gosh, those shoes are just not doing it for us". That kind of thinking is just grasping at straws. It never works that way.
Look at it this way: a couple of years ago I sent a business plan into a local VC group for their paid-evaluation service (pretty good deal for them, huh?). This time, for my $200, I got back a bunch of comments. What did they tell me? That the reviewers were idiots, that's what. And how exactly could I use this information? Except to be bitter, not at all. So not knowing when it's qualitative is a pretty good thing. As for me, I'd rather save my money and get rejected for free.