Multiple times a year, mostly in late spring and autumn, and that's infrequent compared to some people I know.

People use cars differently. Your experience may not be universal just because it is consistent. Besides individual variation people also differ in whether and how many passengers they regularly transport and how many drivers share the car (e.g. my wife and I share one car and often drive together).