I wish they just added a simple "handoff" option in the control panel. I use spotify on all my devices, and use their device handoff a lot and it works really good. Would be perfect if i could open control panel, click audio handoff, and select the detected device from a list.
For example, let's address your concern with my comment.
"Start by listening to something on my iPhone. Sit down and continue to listen to my music on my iPhone. Open up my laptop. It unlocks with my Apple Watch. I pop open iTunes or Spotify, hit play with my Air Pods still in. Since my iPhone is still playing music, Apple should stick with my iPhone."