Yes, it is evaporative cooling. That's why I mentioned the pores. The mud wets and then later the moisture evaporates drawing energy from the surrounding clay.

Phase change can get you below zero easily, it certainly works as a poor man's AC. It is incredible how much energy phase change requires compared to just raising the temperature of the medium.