Now put freshly boiled water in a Thermos, or your preferred brand of vacuum insulated flask.
Notice how the water stays hot for, what, I don’t know, 24hrs or so.
That’s how ”cold” space is.
Incidentally, did you now it is estimated that the Sun, or star Sol, is anticipated to take trillions of years to cool via radiative dissipation after nuclear fusion stops in the core.
Had you thought that your face probably could feel the heat of the Earth being in space? Suppose your skin is resistant to vaccum, suppose you are on some low orbit (so the Earth would have really big angular size), and suppose that you look at the Earth and then turn to look into a deep space so you have no "nearby" objects in your field of view. I believe, that you can feel the difference with your skin's thermal receptors. Even if you are about the night side of the Earth.
I'm not entirely sure about it, maybe I missed something when doing math, but I'm still pretty sure. How can you talk about space not being cold with all this?
Yes, it is. There is such a phenomenon as infrared radiation. You feel it on your skin right now. It is hard normally to separate them from other ways you exchange heat with your environment, but in some conditions it is easy. Like when there is a hot object you can feel it to be hot even if you don't touch it or the air heated by the object. Likewise you can feel that a cold object is cold thing without touching it or cold air around it.
These experiments would be hard to run in space, because it is highly polluted by vacuum and it is not a good idea to expose human limbs to vacuum. But artificial sensors could be used for that.