Patent trolls though. A different story. In 5 years we were sued twice (that I know of) by trolls. On both occasions the trolls had some trivial enhancement to a well known technique. I imagine they are operating a mud slinging operation where eventually something sticks. Once it does then either (1) the cost of licensing this 'new' tech is priced less than legal costs or (2) the troll is looking for testimony from some respected entity that the tech is innovative which gives them validation to go after the next biggest vendor up the chain.
The largest by marketshare, nVidia and ATI (by belonging to AMD) certainly have a sh.tload of patents. Do correct me, but aren't IMG/PowerVR and nVidia holding the majority of the mobile GPU market share anyways?
People who are going to sue them aren't going to be dissuaded by lack of initial source code access.
I can't even remember a case that started by someone looking at source code to figure out if their IP was being used.
It's not inane. Their fear is a natural consequence of mixing ludicrous patent systems with cutthroat industry politics.
Most GPU vendors don't have massive patent war chests of their own. They have a lot to lose by exposing their implementations, because of course they all infringe on various patents held by other vendors. Any program longer than hello.c is likely to infringe on various patents. When there's money at stake, the incentives all point towards keeping the driver sources private.