What matters is that the server or software provided and anyone not in the group can't obtain plaintext even if they can intercept and modify all traffic on the Internet.

Obviously you need to rotate the session keys when someone leaves, since they must no longer be able to obtain plaintext.