Well, seatbelts aren't the be-all, end-all, either... And people have indeed complained about them when they were mandated. So, it's not like "one could say the same thing about seatbelts, so look how absurd it is". Someone else might easily reply "my life, my choice" about wearing one (same as for drinking, smoking, over-eating, etc).

But even so, we can consider seatbelts as a necessary wearable/accessory for car use, which is also a recent development (cars and driving). Masks are different in that they affect people going on about doing things people did for millenia without requiring a mask.