The game actually tracks this too, what is visible for each player, which gets slightly complicated with diplomacy changing. Of course this doesn't prevent you from just patching other parts handling this, but you cannot just simply modify this value, it will desync as well.