How is this any different than saying "Cell phones are too complicated so lets just skip all that research for practical reasons?"

I agree that many patents are held by groups that don't use them how we'd like them to, but they still had to _buy_ the patent. Society promised them that the patent would be enforced and it is. Combating abuse of the courts is a separate matter.