Writing OS's is hard and there are no shortcuts. You should take a look at microkernels and the work Andrew Tanenbaum did with Minix 3 [1] and his modular design as this is the sort of direction you're probably looking for with customisable OS's.

[1] https://www.youtube.com/watch?v=bx3KuE7UjGA