I think I'd be happy with a stable API, it wouldn't have to be a stable binary interface. Wanna change the internals of a struct? Totally fine. Not make binary shims for 32-bit drivers? Totally fine.

That would encourage open-source or at least source-available drivers (to allow recompiling for ABI changes), but recompilation should be straightforward and possibly automatic if the API doesn't change every couple months.