Using "gay" as an insult is very problematic and homophobic.

Calling someone who is gay "gay" is generally not a problem. (Depending on context it obviously can be, but that's then more dependent on the context, not the word.)

The implication of calling someone who isn't "gay" is that being gay is somehow bad or something to be offended about. That is as clear a homophobic statement as there can be.

It is homophobic, by the way, even if the intent of the speaker isn't there.

Perhaps, but construction workers tend not to spend much time considering the implications of language in the way that people who are college educated might. So what you have is more a clash of the cultures than real homophobia.

The way I've always thought of the word "gay" being used in that way was not so much that it was bad to be gay (as in the sexuality) so much, but that you are associating the person with certain stereotypical gay traits such as effeminacy which would be somewhat insulting to a stereotypical macho construction worker.

As I already said, that's also sexist. Gender policing is not ok. (And you cannot ignore what gay usually means.)

That depends, if you are in an environment the person on the receiving end doesn't take offence and will happily give insults back then it doesn't really matter.

The difficulty is when you put people not used to that environment into it, the real issue is a conflict in expectations rather than the exact language used.