But drivers for custom hw are easy to rebase. They are quite self contained.

It is when you get into kernel internals that rebasing continuously becomes a challenge. Some parts of the core Linux kernel don't change often, but I imagine many other parts see significant churn.