This is because a bunch of the units on the market source their heat from directly around the water heater (likely a conditioned area of your house). They have options for ducts to dump the cold air they generate outside during the winter, but this doesn't help as you then need to draw in outside air to make up the pressure differential (otherwise good luck opening doors).

Units that have the radiator/compressor outside are 3-5x the price, and are hard to find.