Just to jump in here, there's also a measure of agility that is required. Having any sort of formalized system based on criteria assumes by its nature that those criteria are always going to be true. Anybody who's been watching the tech world for more than five years knows that's not true.

Not to mention the issues that come up once the system is changed. If the system is changed next year, and some companies that failed under the old system would pass under the new system, wouldn't that be a huge problem?

You're taking people at the top of their game, with knowledge in the industry and technology, and having them make a fine-tuned judgment call.All this for a dynamically changing business environment. If the decision is of a well-considered and thoughtful nature at all, that's going to be very hard to systematize. The more you formalize it the more you lose your special sauce. There are qualitative scorecard-type solutions, but those by nature are personal opinions and would have to remain proprietary.