Not getting into the tree is a problem because kernel interfaces change all the time. When someone changes a kernel interface they are expected to update all of the affected code, but out-of-tree drivers don't get that.
This leaves Google with a version of Android that does not run on anything. There are less than a handful of relevant SoM manufacturers that are capable of delivering consumer grade SoMs capable of running hardware accelerated Android; Google can not alienate these.
But they don't care enough about this sort of thing (unlike Apple), and no one (such as Google) is forcing them, so it won't get done unless they see an economic upside.
It has worked quite well for those that tried.
I would think they would start to take things more seriously at that point.