I'm having the same issue with Duo. it pauses the video frequently. I switched back to FB Messenger, which just works without pausing video.

Google had said something about Google Duo being bandwidth friendly and good for slow connections but FB Messenger is miles ahead, at least in my usage. Which left me quite disappointed with Duo, as I like the idea of using Duo over FB.