Voice search is basically like Apple Spotlight. It's system wide.

I only expect it to be localized within the app I'm in when I'm in the search box for that app, in which case I'm not using voice search, I'm using voice recognition to fill in the contents of the search bar.

Outside the context of voice recognition for an input, to me clicking the voice button on my apple TV is opening Siri, just like "Ok Google" or "Hey Alexa"