Oh yeah, there's no shortage of reasons not to use SF6. Even in conventional waveguides, as far as I know most designs these days prefer nitrogen or dried atmospheric air.

Anytime i see fluoride, im immediately concerned about anything in its immediate surroundings...

Some fluorine compounds are quite stable. SF6 has two downsides: its GWP from very great stability in the atmosphere (it needs to rise into the mesosphere to be decomposed, not just the stratosphere, giving it an atmospheric lifetime of as much as 3000 years) and toxic compounds produced when it finally is broken down by energetic processes like arc discharges.