The reality though is that failure of the rocket doesn't mean the reactor or fuel is vaporized and thrown to the wind. It's possible to construct the payload fairing to survive sudden, rapid disassembly and crash land intact off the Florida coast for recovery.
A nuclear-armed intercontinental ballistic missile is visually identical to a launch rocket carrying a peaceful payload to LEO. The neural association between "nuclear missile" and "rocket with nuclear reactor payload" is very strong.
It doesn't matter how many facts you throw at it. In the mind of the public, all launches have the possibility to result in another Apollo 1 or STS-51-L (Challenger) or STS-107 (Columbia). Every "nuke" is the Castle Bravo test. Every reactor is Chernobyl. And the public generally is terrible at risk assessment and actuarial math.
You could actually make the launch safer than an afternoon stroll on a Florida golf course, but no one is going to lie down in front of your golf cart screaming, "lightning strike!" or "angry gator!" or even "cardiac infarction!" or "terrorists!"
People may have no problem at all if you just call it "an NCG power plant" (for neutron-cascade generator) and say the details are classified for security reasons--that "security" being the type associated with a toddler's blanket.
[1]: http://www.popsci.com/plutonium-238-is-produced-in-america-f...
Now, as soon as you turn the reactor on you start turning the relatively safe U235 into all sorts of nasty things with half lives that are much, much shorter than 700 million years like strontium 90 with a half life of 30 years. That stuff is indeed horrible and getting it spread over Florida would be a disaster. But the solution is to make sure you don't turn the reactor on until it's safely in orbit.
That makes them much safer than the RTGs we've been using to power space probes far away from the sun previously. Because the P238 isn't in a critical mass it has to be a synthetic isotope with a low half life in order to generate enough heat to power a spacecraft when it's outside a reactor and would be quite deadly if you interacted with it without shielding.