Also, it's really nice (now that Hulu+ has finally allowed us to "cut the cord") that everything is on one (simple) remote now, with no input switching. The only thing we use the Harmony Hub remote for anymore is turning the whole system on/off (my Receiver doesn't seem to do the HDMI control pass-through), or switching to the PS4 or WiiU.
The "stick" devices are nice in their own way. But having a physical remote and a single "all you can eat" device really adds to the simplicity IMO.
It's not so much that it uses the phone as a "remote", since you're browsing the content you want to play directly on the phone. It basically moves all browsing to the device, which they're pretty good at, and leaves just display to the chromecast. I usually prefer it to clicking around on-screen with a hardware remote now.
It's the same reason the WiiU app navigation is a big minus for me. I don't ever want to see that Mii lobby thing. Don't know what the hell it's for. Just put the apps on the TV like every other console. Mirror on the gamepad if you want. That's fine. But don't try to force it as the default option to the exclusion of the rest of the party. (Yes I realize you can "cast" it up to the TV, but is it sticky? I don't know, because it still seems to routinely annoy by going back to the gamepad.)
Anyways, yes, the Chromecast has it's place. And it's a nice fall-back when the AppleTV in the conference room is unreliable because you've got a dozen of them around the office, 3 different WiFi networks nobody knows which to be on, and the AppleTV disconnects every five minutes while your laptop doesn't realize and getting reconnected takes a minute or two and ugh it's just so frustrating... Never have that problem at home.
Lots of users want simplicity on their TVs, not power. The Apple TV is easier to set up and use than a PC. (When I looked a number of years ago, it was also quieter and drew less power, but PCs have probably caught up.) The experience of using a keyboard and mouse is also awful from a couch without weird trays or additional furniture. A $150 Apple TV is also cheaper than a $200 PC + keyboard + mouse.
For an appliance that plays YouTube, Netflix, Hulu, streaming music, media from an iTunes library, and can stream audio and video from your phone and computer, the Apple TV works pretty dang well.
That said, I have a fourth generation Apple TV after previously having owned a third gen. The fourth gen has a lot of potential, but has not yet pushed TV forward. I've used the App Store mostly to get new sources of audio and video to stream. I haven't yet found any killer app interactive experiences that are more compelling than they would be on other screens. Maybe that will end up being games, but it will probably be something nobody's thought of. (Personally, I really hope, maybe Apple, figures out a great web browsing UX for the TV, but I'm not counting on it.)