A lot of sweaters get sold at Fisherman's Wharf during summer months to people caught unaware.
Local weather is something you experience every day, not something you travel to. Same goes for snow and tahoe.
We lived in the Sunset district which was manageable but later moved to Mill Valley and the summers were _very_ brutal, to me. A couple other office mates lived in San Rafael and I distinctly remember everyone complaining about the heat and sweating.
We all eventually bought air conditioners for our bedrooms just so we could sleep comfortably at night.
But brutal is all a relative term. Someone growing up in the south with no access to AC would not call it brutal while I and my other office mates who were used to AC were "dying" without it
Most of the microclimates in the Bay Area are disgustingly peaceful, but as the parent notes, there is a distinct lack of seasons (there's approximately two: dry summers, wet "winters") and real weather (I greatly missed thunderstorms when I was there).