Fresh is there at least partly because cars didn't used to have AC. If you are driving in-town or stop-and-go having a fan on your face with fresh air takes a bit of the edge off.

Also in heating mode, there's little reason not to use fresh, as we can heat up cars really easily.