I usually see 'bad culture fit' to mean either... too old, too independent (not submissive to 'company culture'), not quirky enough, not upper middle class enough, or too arrogant.

I think existentially I would feel a lot better if I got nixed because I was too smart, rather than, for example, not being lucky enough to be culturalized in an upper middle class setting.