Hmm. I was using a surprisingly similar setup (Arch, Sway, and Firefox) and never had those issues, but I think I was using Firefox with XWayland instead of those environment variables. I'm glad it's not a huge pain for you.