The lack of a stable ABI does not prevent maintaining decent drivers, hundreds of drivers have been up-streamed onto the kernel and work fine.

The lack of a stable ABI makes out-of-tree harder to maintain, but the whole point of Linux is that you don't need to maintain them out-of-tree.