I don't think that's the case at all. I think its workers who have seen the job situation get progressively worse in the US over the last few decades, and are trying all avenues to prevent it from getting worse (in this case, depressed wages).
If workers had more protections (as it is in other developed/first world countries, for example), I don't think you'd see such a strong backlash against H-1B visas.
The H1-B is a "dual-intent" visa -- you can be on it while pursuing citizenship. This differs from the TN status available via NAFTA, for example.
Additionally, the H1-B has a 6-year limit in theory (3-year limit, renewable once), but can be extended past that limit if the holder is at a certain stage in the permanent resident application.
Most of these workers strongly desire to immigrate to the Untied States on a permanent basis, and the truly skilled ones (I'm not talking about ones hired by outsourcing giants) work really hard (and usually get paid really well), and contribute significantly to the company (and as such, to the United States as a whole).
(And certainly the temporary status factors into the fact that they exist and how many are granted and all of that stuff. FWIW, I think the U.S. immigration policy is a disaster, we should be letting far more people in and looking for ways that we can help to improve the situation in Central America (because presumably that will reduce the most common sort of illegal entry).)
The pathway to a green card (permanent residency), for nearly everyone else (i.e. skilled immigrants) involves getting an H-1B or L-1 first, and then asking their employer to sponsor them for permanent residency.
So when I hear anti-H1B rants, what I'm hearing is hatred towards immigrants and immigration in general, that seem to be coming from people who have forgotten their own heritage.