Multiple times a year, mostly in late spring and autumn, and that's infrequent compared to some people I know.

People use cars differently. Your experience may not be universal just because it is consistent. Besides individual variation people also differ in whether and how many passengers they regularly transport and how many drivers share the car (e.g. my wife and I share one car and often drive together).

Probably because you have an expensive enough car, that actually does a good job at regulating the cabine temperature.

In my cheap pickup I have to constantly adjust temperature and fan speed. In my old German "ultimate driving machine", it's set and forget.

Constantly.