The underlying work you guys have done is great as a proof of concept, and after playing with it for a short while there are some obvious flaws and improvements which basically just come down to user experience.

As it stands, everyone's inputs clash.

The main issue is there's a single mouse pointer and keyboard input that all users control. It's quite chaotic as people scroll, click, select, etc. Provide some mechanism to nominate control authority or vote on interaction and it might be usable.