Sometimes information leaks (most well-known example are the leaks of Snowden) or hints come up.
One example: https://theintercept.com/2017/05/11/nyu-accidentally-exposed...
A (German) commentary on this article: http://blog.fefe.de/?ts=a73ff836
It's not just the NSA, it's literally everyone else as a class of threat they might need to consider. Also, I use opposition researchers as threats for politically exposed people, and who cross over into foreign spy level stuff.
The controls it prescribes are straightforward, and realistically, it's a risk you just understand, do your best to mitigate it, and accept.If you are going to not do business because you are afraid of state level consequences, you've got a legal/regulatory problem, and not a technical one.
They could know more than civilian cryptographers, have new direct attacks that we don't know yet, e.g. algebraic attacks and specialized hardware to solve gigantic systems of equations. Or, they could have a working quantum computer with many qbits. We don't know, do we?