I think it works only if everyone is playing the same chord/scale. Kinda like someone playing with a loop pedal. You're right that it definitely wouldn't work with a piece of music.