Private keys are marginally more complex, but not much. If a password is sufficient security, then the private key can be stored remotely (in S3 or whatever) but encrypted (symmetrically) with a password.
So say Alice wants to share a file with Bob. Alice's client downloads her encrypted private key, and prompts Alice for a password. The private key is decrypted with the password and stored in memory. Alice then downloads the file she wants to share with Bob, and decrypts it with her private key. Then she downloads Bob's public key, and re-encrypts the file with Bob's public key. She can now send the file to Bob securely without the server being aware of the content.
Adding members to the group is trivial; just send them the group's key pair. Removing a member would be more difficult. Perhaps the most convenient way would be to add an additional layer of security on top (so members would need server access permissions, plus the private key). The only other option would be to create a new group and to re-encrypt all the existing files with a new key.
AViD's answer on this security stack exchange is useful.
(http://security.stackexchange.com/questions/406/how-should-i...)