Your system is a bit of a black box, so I'm just going by my best guess here. I'm imagining that after the obviously great applications have been added to the interview list, and the bad applications have been tossed, that there will be ones in the middle that require discussion and debate between reviewers.

If I was a reviewer, I'd be annoyed if I spent a bunch of time debating the merits of an application with another reviewer and discovered we'd been looking at completely different versions, submitted a month apart. "This application is awesome, they clearly have what it takes as founders and have a cool idea with a prototype." "What are you talking about? That application was one of the most confusing I read. And it was just an idea, no prototype." "Oh, we were looking at different versions. Damn."

It sounds like it isn't a problem though, so maybe this situation is rare, or for the first cut you don't discuss at all and just go by pure score averages, or there's something else I'm missing. Thanks for the feedback.

If two people were debating the merits of an application, they'd look at it, and they'd both see the latest version.

No one would try to talk about an application without it open in front of him; they're too complicated and there are too many of them.