That sounds plausible as a rationale. In my experience "bad culture fit" has been used as a stand in for "I don't want this person working here and I don't want to explain why" about 80% of the time.

The reason is often racism. This time it could legitimately be illegal union busting.