Everything beyond that is the precautionary principle.
It's also really important to understand the difference between Dual_EC (the random number generator) and the NIST curves. There is much more circumstantial evidence against Dual_EC. Importantly, the potential backdoor in Dual_EC isn't really related to elliptic curves; you can describe a functionally similar backdoored RNG using other public key algorithms.
One fortunate result of the Snowden disclosures is that for several reasons, some rational and some irrational, the market value of NIST/FIPS certification has plummeted --- it's still an issue if you're selling to the government, but no longer carries security cachet.
As a result, there's minimal upside to adopting cryptographic primitives and constructions simply because they have NIST standards backing them. Which means there's minimal upside to using the NIST curves.
Meanwhile, there are multiple downsides. One of them is the potential for backdoors, but I don't need to reach that issue in my analysis because another is the difficulty of safely implementing curve software with the NIST P-curves.
...until some worker or contractor takes their "secret" values for himself, or sells them, or publishes them on the internet. Producing the public standards with the built-in master keys increases possibility of overnight global breakage.
See: https://en.wikipedia.org/wiki/Nothing_up_my_sleeve_number
(Or, of course, you could just not publish RNG standards based on public-key crypto ;-)
It's believed that any elliptic curve algorithm that doesn't have a transparent process for choosing the curve points may have been backdoored by the NSA choosing points that they already knew how to factor. If you use those curves, then you're revealing your secrets to the NSA but not to anyone else, because the discrete log problem is still (mostly) just as hard as it ever was.
Specifically, the elliptic curve random number generator in NIST SP 800-90A is believed to have been backdoored by the NSA. For obvious reasons no one has any hard proof, just very strong circumstantial evidence.
You can continue to use SSH2-RSA with decent size (2048 bit as a minimum) keys & AES. Those are not believed to be breakable at the current time, although as ever you can never have absolute certainty in these matters!