Not getting into the tree is a problem because kernel interfaces change all the time. When someone changes a kernel interface they are expected to update all of the affected code, but out-of-tree drivers don't get that.
This leaves Google with a version of Android that does not run on anything. There are less than a handful of relevant SoM manufacturers that are capable of delivering consumer grade SoMs capable of running hardware accelerated Android; Google can not alienate these.
But they don't care enough about this sort of thing (unlike Apple), and no one (such as Google) is forcing them, so it won't get done unless they see an economic upside.
It has worked quite well for those that tried.
I would think they would start to take things more seriously at that point.
New phones need new drivers which must support the new kernel, period. I don't see what the big deal is. You aren't getting Oreo on your old ass Samsung Galaxy S2 anyway.
We're talking input controllers, wifi, bluetooth, nfc chips, gyroscopes, amps and half a dozen other parts that never attempted to have drivers mainlined in Linux kernel.
It's a chicken and egg problem - they won't make updated blobs until Android doesn't include newer kernel. Android won't use newer kernel because that would block 2/3 of the phones from being upgraded because of Broadcom bluetooth chips alone - Samsung might decide they are better off forking Android than waiting for Broadcom.
Edit: Sony has a bunch of phones that can run on 4.4 kernel: https://developer.sonymobile.com/open-devices/