But they’re also just individual atoms. They’re not a chunk of radioactive material that can sit on your roof for years.

Besides, the whole ship would have maybe 15kg of fuel. That means that there can only be a maximum 15kg of fission products. It would burn fuel that for several years, spreading the exhaust out over the distance from here to Pluto and back.