Heat conduction rate is linear in response to the temperature difference [1], so 1 day of 20 degrees difference should give the same total heat loss as 2 days of 10 degrees difference. They also both give 20 degree days, so degree days is the correct measure and should correlate linearly with the energy needed for heating or cooling. The same goes for the heat content in the air that is exchanged due to draft, opening doors etc.
You might have a little bit of a point with sunlight, and humidity. But my guess is that they don't dominate, compared to the conductive and convective heat exchanges.
[1] https://en.wikipedia.org/wiki/Thermal_conduction#Fourier's_l...
I think my comment about mass transfer when opening doors could maaaybe be defensible but I'll leave it alone :)