The NVIDIA driver has been long subject to exactly this controversy, which is one reason why distros don't like to ship it.
Many people think it is, but apparently no one with authority to go to court over it has done so.
It certainly would be interesting to see a "Software Freedom Law Center" vs "Free Software Foundation" legal battle. But if no one wants to fight over a clearly more egregious graphics blob, then why would they go after Canonical for something seemingly less of an issue.
So then the question is, why is "nvidia.ko"'s binary blob not a derivative of Linux? What's the logic that says one binary blob is a derivative work, and the other is not when they both are 'ELF binary with Linux metadata and Linux module entry points'.