The only time I use the actual remote for my TV is when I want to fiddle with the TV color & brightness settings, or switch to an HDMI input with my laptop (but actually usually I just press the input button on the TV itself in that case), both scenarios are very infrequent; otherwise I basically exclusively use the Apple TV remote, it can turn on and off my TV and otherwise everything just happens inside the Apple TV UI.