There are a few devices floating around with a hardware switch built in. If you use a Pixel, grapheme OS is probably pretty trustworthy so you at least no there's nothing nefarious down to the OS level.

But yeah, in general if you take a phone just assume it's tracking you or at least making it possible for those with access to know you where there.