IR sensors don't work by detecting heat changes. They convert infrared photons directly to electrical current.

Now I don't actually know jack about IR sensors, but I would assume that increased temperature means increased noise, as it does with visible-light CCDs. People refrigerate visible-light CCDs too, for taking low-noise astronomical photos.