If someone gets to pick who's allowed onto a network, then they won't bother using it. Maybe your friends will join since they are allowed to add _their_ friends, but those friends of friends wouldn't bother because most of their friends can't join, meaning your friends won't bother either unless they really want to talk to you specifically.

It would work for groups where most people know each other, but there are already options for that that allow users to be in groups that aren't all owned by the same person e.g. Discord, Signal, Facebook.