The same remark can be applied to other fields; recruitment, record companies, auditions to cast a role, men/women relationships, .... what's common here is that, although you can tell success or failure retrospectively, you cannot nail down the systematic criteria beforehand.
Not to mention the issues that come up once the system is changed. If the system is changed next year, and some companies that failed under the old system would pass under the new system, wouldn't that be a huge problem?
You're taking people at the top of their game, with knowledge in the industry and technology, and having them make a fine-tuned judgment call.All this for a dynamically changing business environment. If the decision is of a well-considered and thoughtful nature at all, that's going to be very hard to systematize. The more you formalize it the more you lose your special sauce. There are qualitative scorecard-type solutions, but those by nature are personal opinions and would have to remain proprietary.
Also, it would be great to see a YC applicant actually develop a system for the applications themselves making feedback and follow-up easier for YC staff.
So, if systems were in place to handle these types of apply-check-judge-accept/reject-feedback-followup processes better, I'm confident better outcomes could be reached.
Anaphoric's comment (http://news.ycombinator.com/item?id=68979) such a system could be used by that manager who rejects at random.