Agreed. This would be only slightly annoying if you're talking to the person. But 100ms or even 200ms is an acceptable latency, especially if it's constant. This solution is not just simpler, but it's also more efficient as you can bundle the data up efficiently, and include state changes as well.

There is a lot of prior art in this space, btw. I remember Meteor.js having a great real-time demo over websockets that actually used predictive techniques to keep things (imperfectly, but still impressively) 0 latency.