Building a kernel of your own is fairly trivial. The potentially less trivial part is keeping it up-to-date with security fixes, etc. Not impossible by any means, but it does require a bit of a commitment in time, build environment, etc.