FOSS NT Kernel... It would only be successful when it is really FOSS and not company backed single main contributor FOSS like Chrome or .NET Core. As a kernel contributor I want to shape my kernel, not being overruled by a decision done in a meeting somewhere on a campus I never visited.
You would need extensive partnership agreements like what Google has with Microsoft regards Chrome or Samsung has with Microsoft regards .NET Core. Challenging for smaller vendors.
Linux has a more pure (not owned) FOSS style making it successful.
I agree a significant part (necessary part?) of Linux success among hackers and developers is it being FOSS... I don't know much about NT kernel but it would be quite interesting to see the alternate reality in which it was also FOSS and independent of the rest of windows, I wonder if it would have a similar success in the arenas Linux does.
There are some other really interesting FOSS kernels out there today like Minix3 (quite different from the Minix that inspired Linux), they are sure to have deficiencies when directly compared to something like Linux due to the difference in the shear size of community driving them, but they have noteworthy designs, superior in various aspects, and they make me wonder, is there only enough space in the world for so many popular FOSS kernels? i.e how much of Linux is now inertia much like windows desktop? (and I say this as a full time Linux user)