People criticize the patent trolls that are using generic/obvious patents that somehow got approved when they should have been rejected.
"Interface for opening a door by use of hand and twisting motion" (aka: a doorknob)
Those types of patents. The ones worded ever so carefully as to try and sneak by being declined. When people refer to patent trolls they are mostly referring to the companies that purchase the rights to these kinds of patents and go on a litigation hunt of any big fish they find infringing to try and sucker out payments.
I do not see it as patent trolling when someone who say... invented a jetpack and has a patent on the technology in use going through a firm to sue an infringing superbody who has infringed upon the patent to sell their jetpack because they can mass manufacture and undersell the small business. That's what I see as a "legitimate use" of patents.
On the other hand, "Backpack to be worn to fly through the air through means of jet propulsion technology" (aka: jetpack with no working functionality patented) is not a legitimate patent in my eyes and is sat on to prevent anyone from making jetpacks because the moment they invent a jetpack they'd be sued. That's what stifles innovation.
I will agree that there is a substantial patent quality problem, with a number of overly broad and poorly-defined patents being asserted by firms, non-practicing or otherwise. However, from my observation over almost a decade, I would also say that this problem is declining rapidly, as most of these patents were issued during a period when the PTO had no idea how to deal with them two decades ago.
This is the exact problem that many patent trolls solve. If your patent is strong enough, they will take your case on "contingency" and take a (huge) cut of the winnings, if any. Patent trolls can help even the imbalance between the big guys and the little guys.
Not surprising then that the loudest complaints about patent trolls come from the "big guys". Unfortunately for them, multiple studies, including by the GAO and professors like Michael Risch and Coleen Chien show that there is no real patent troll litigation crisis. Fortunately for them, tech media is happy to regurgitate their PR for the rage-view value, and most people are happy to accept it uncritically.