I wonder how frequently this "stain" impacts people in practice. I've got some defense contractors on my resume. I suppose some people might flinch on that, but I've never felt like I had a problem getting work or growing my career.

I suspect it might be like that. A small percentage of people would nix you, but you would never know. Compared to something like the porn industry, which I've heard is death to a resume.