I agree wholeheartedly, the only downside I would see is that volume cannot be controlled through the iPhone app if you're controlling the volume through the TV. When I have a bluetooth speaker paired to apple TV, i'm able to control that volume using the rocker switches on my phone, it's the weirdest thing.

It feels like magic to have my TV turn on by a touch of my phone, and to also be able to control the TV when the kids think they are sneaky by not giving me back the remote.

My sound bar uses ARC connected to my TV. This means the physical Apple TV remote can turn the volume up and down. Same applies when using the Remote app. The volume rocker on my phone controls the sound bar volume.

Annoyingly, this does not apply to screen mirroring to the AppleTV.

I built a raspberry pi IR blaster to solve this very problem. I have shortcuts on the home screen that talk to the pi when I want to control the TV volume from my iPhone.