Casting from my phone when I sharing a TV with other people who may not have the app installed, may not have an Android phone at all isn't more convenient than just handing over a remote - I'm not going to just hand my phone over silly nilly to let someone else control the TV while I'm out.
I'm not suggesting that most people get an AppleTV 4K - they are overpriced and I got one free with my DirecTVNow subscription - but at least with it, you can airplay Google Play and you have native apps (or Airplay) with Amazon Prime.
With Roku you have native apps for everything except Apple's stuff and you can cast to it.
With the Nvidia Shield - you have access to the entire Android ecosystem, Amazon Prime, and ChromeCast support.
The ChromeCast is the least capable alternative.
This isn't even an anti Android thing. The Nvidia Shield is Android based and just much better.
Uh, yes. Yes. I am really going to say that. Again, you either haven't tried a Chromecast, haven't ever played a Netflix[1] video on an Android phone, or both.
Again: yes, playing content on a phone is easier than a TV. And it isn't even close.
Did I mention the phone is easier? It's easier.
[1] Or Google Play, or Youtube, or Spotify, or pretty much anything except Apple or Amazon, which remain notable and regretful (if not unexplainable) holes in coverage.