In a lot of cases, the OEMs publish their kernel source, with their drivers and changes, so it's not a GPL violation. But it takes a tremendous amount of work to get that kind of thing into the mainline kernel, not least because of the atrocious quality of a lot of OEM kernel code. And the incentives are absolutely not there to get them to do it.

(Of course, some OEMs absolutely do violate the GPL, and then there's also the issue of binary drivers and blobs that violate the spirit but not the letter.)