I believe the main problem is that Windows goes to reasonably extremely lengths (it's not perfect, but it's fairly good) to not break the driver interface, so if someone makes a driver for something, then semi-abandons it, the driver will continue to work for many years.

This is often clear in graphics cards, the main problem is old cards don't get support for newer versions of DirectX, but otherwise continue to work.

Linux explictly doesn't support this -- that means when some hardware manufacturer makes a closed source driver, it's very hard to update the kernel. Unfortunately it seems many bits of important phone hardware (like stuff Qualcomm makes) only have closed source drivers.