From what I understand there are a number of reasons.

One of them is that domestic oil and gas production was on the decline until the tech for horizontal drilling and fracking started the shale boom of the last 10-12 years. Combined with demographics (big boomer generation, small generation X, big millennial generation) this means you have to find and train all these millennials who probably had other plans. And then, I'm told, the highly cyclical dynamics of the industry make any kind of long-term investment difficult, so there's high overhead for a labor market that still relies on fax machines.

If you're macro-minded and you want policies that keep that labor price high so alternative energy becomes more attractive, something like the Green New Deal is probably a good bet.

The pay is good but working conditions?

Then the pay is not good enough.

The pay is good enough but their are other constraining factors when it comes to extreme rapid growth in an area. Housing shortages are one example.

True, but that's part of the issue. Volatility affects people's lives, and they need to be paid extra to deal with it (assuming they have other options). Moving around a lot, living in the middle of nowhere, inconsistent income due to changing markets, these are all negatives that need to be compensated for in the wages.