Wearing swag is as much as a signal as wearing "fast-fashion" clothes or insisting on wearing a shirt and tie on places that do not require it.
... for you.
Many are more utilitarian with clothes. Clean, comfortable, and suitable for the activity/environment are most important. Color, pattern, design is more of a minimum threshold filter (e.g not going to wear something offensive)
It's similar to food. Some people "eat" because calories are required and they don't care much being that, while others "dine" because they want something more out of it than generic calories.
Filtering is still self-expression and related to social dynamics. What you don't say can be as informative as what you do.
If that marks me as "uncool" to certain people, that seems like an added benefit, as i can't imagine i would get along well with anyone who judges people on such a shallow metric.
Vast majority of people are genuinely just wearing some clothes and are not signalling anything.
If you can see that the person is pondering about the answer, it tells you right away that they do care.
Similarly i might not care generally, but there is a line where i start to. I wouldn't wear a shirt with a picture of dead babies on it. Does that mean i care? Maybe technically, but in context i dont really think it shows i care in the way that was implied originally.
[0] https://www.harmontblaine.com/US/all-the-good-things/t-shirt...
[1] https://www.redbubble.com/i/t-shirt/Elf-Green-and-Black-on-W...
How can it be that so many people say they "don't care" about what they wear, yet their overall style is so uniform?
That kind of goes in with "not caring". I agree with your overall thrust here - clothing is a social signal whether or not you want it to be. Even the lack of someone signaling is in itself a signal. It legitimately took me a very long while to figure this out.
For me - comfort is all that matters, even in public, until I get into a social situation. Then it's simply dressing to "the average" as to not stick out. I used to describe this as not caring, but it definitely is caring in my own way. Plenty of other folks dress specifically to stand out, so the opposite is certainly a positive action as well.
In the end, me simply wearing pretty much the same dumpy thing everyday sort of became a personal brand in of itself.
That said, I'd be perfectly happy with everyday uniforms assuming they are comfortable. It would save me mental effort of having to imagine what other folks will be wearing in a given situation.
Not caring in that context means minimizing effort, time spend and external impact on their lives, while maximizing comfort. Buying what everyone else buys maximizes that value.
I would say I don't care about clothes, but had to make some choices when I got cats. I used to wear primarily black clothes, but cat fur is extremely visible and hard to get off them, so now I just buy diverse colors, while still not caring whether the green is too bright and goes with beige or whatever.
Like i might not care generally, but part of not caring is that i don't really want to draw undue attention to the item in question. E.g. I probably wouldn't wear a shirt that was neon green sequins or something similarly ostentatious. That said, im not going to care about a pocket one way or another.
I suppose you could say not caring is a spectrum not a binary.
None happen to be emblazoned with my company logo (and we make swag), because the better swag we give out are hoodies and exercise pants.
(Small shoutout to Cloudability here, who gave away some of the most comfortable swag I have. I’ve likely worn their shirts (2) almost 100 days in total.)
They're just clothes! Most people aren't that much into fashion that they'd care like you think they would!