You could switch to a ski helmet, those have removeable insulation pads. Otherwise a tight fitting thermal cap should do if you are huffing and puffing.