Is it possible to have a material that doesn't conduct heat?

No, that would essentially mean that it doesn’t interact with neighboring matter (eg air bouncing off of it) since heat is really an emergent property of all the particles in a material bouncing/vibrating amongst each other. There are things that don’t conduct heat but they aren’t regular matter.

No, but you can get surprisingly close with aerogels:

https://www.youtube.com/watch?v=PmKt_6EhA24

The lack of a material, e.g. a perfect vacuum, as long as radiation doesn't get involved.

Radiation becomes the dominant heat transfer mode in that case though, usually gets really small at day to day temperatures though.

It's not that small. Blackbody radiation at room temperature is 450 watts per square meter. And it only takes an extra 15C to get 100 net watts transferred per square meter.

Radiation is always involved, even at very low temperatures.

Superconductors come pretty close.