That's how the ranking of HN stories works now. It's a combination of upvotes and human curation by users (e.g. flagging) and moderators.
So there's going to be some combination of technical and non-technical factors. How exactly will that work? We don't know yet. Let's see what happens and figure it out together. That's part of the experiment: we'll figure it out in the open and adapt to feedback. But let's wait until we have some data to look at.
FWIW, my experience with HN is that genuine community interest is relatively easy to recognize.
Another thing to watch would be the mean user karma of all votes. A bunch of low-karma vote stuffing would actually bring a post down.
If you can get a ton of massive karma users to upvote and game this, maybe you should just get in by default?
More interesting to me though with all this is what it say’s about the future of scaling YC. YC’s mission is to enable innovation, and funding more startups hopefully leads to more innovation. The full YC program, already spread across two demo days, is probably close to the limits of how many startups can be funded in one go. The Fellowship, by being virtual, is much more scalable but one of the key bottlenecks is always going to be reviewing applications – this experiment could help remove that bottleneck. (another bottleneck I see is office hours, but getting sci-fi for a moment maybe YC research can develop a chatbot for that one day!).
High uncertainty is actually good. That's what makes experiments valuable - they reduce the uncertainty and thereby help us better understand something. In this case that "something" is the HN application process.
So even if this process has potential problems, the experiment is still worth it.
A) There are many external factors that can implicitly cap the number of upvotes/comments. (time submitted, amount of competition, etc.) HN has repost rules to alleviate this problem: would Apply HN posts be able to repost too?
B) Not to mention that it encourages sockpuppet voting/commenting, especially since there is a high reward for doing so.
Product Hunt, for example, thrives on "how can I get exposure for my startup submission outside of the intrinsic quality of the startup itself?" and it would be an improper fit for HN.