I can believe a small group can exist as a collective without violence for a while (maybe a long while), but how can any such group deal with a member that decides to use violence to get their way? Either that person becomes leader, the group organises to stop them with violence or the group dissolves.

And I don't think even such a temporary arrangement can scale much above Dunbar's number.