That's true, but most rooms where this is a problem (the huge ones) aren't encrypted. There's no way you're going to cross-verify everyone in a room with 4000 users in it so encryption in such rooms is essentially useless.