The Kilopower design is not an RTG (Radioisotope Thermoelectric Generator). Those rely on the decay of a heavy element like plutonium to generate heat, which then generates electric power using a thermocouple.
Kilopower is making an actual fission reactor, which generates heat from the chain reaction of fissile atoms splitting each other. Another difference is that their design will utilize a linear-actuating sterling engine to convert the heat to electricity.
That being said, you're correct. The management of thermal flux and particularly discharging waste heat is hugely important to keeping this design working and efficient.
https://en.wikipedia.org/wiki/Radioisotope_thermoelectric_ge...
Kilopower will utilize highly-enriched uranium fuel to power a fission reactor. This means that the atoms in the fuel will be sustaining a chain reaction of fissioning, releasing neutrons, causing more atoms to fission, and on and on. This is what generates heat in reactor systems. This design will prove usable for shorter, higher energy demand missions.
https://www.nasa.gov/directorates/spacetech/kilopower
An RTG is essentially a lump of fuel which sits and decays, generating heat. Kilopower's reactor would be an actual reactor, needing to manage reactivity, burnup, and maintain criticality. They are vastly different systems. For someone not in the field I can see how there'd be some confusion but please be less speculative on topics of which you're less familiar.
...but the cooling part is critical. These are RTGs which basically just use the heat generated by the nuclear material to generate electricity through thermocouples which rely on the difference between the hot (plutonium) side and the cold radiator side. Without sufficient cooling the power output will be much lower because the temperature difference will be lower.