Note the "more importantly" - one of the reasons the conversion machinery is bulky is because you need all that space for radiation shielding anyway, and the conversion stuff tends to serve dual purpose.
Active nuclear (fission or fusion) reactors put out a lot of radiation, much more than even high-level waste.
Sounds like pretty much what I would have expected in terms of limitations, so we'd need something much more energy dense (Like fusion) and a more compact way to remove the heat and convert it to power.