Typically, the scale of embedded systems runs from very BoM cost sensitive (i.e. making millions of the things), to very cost insensitive (i.e. tens of high value items).
Microcontroller-based systems tend to be on the very-cost-sensitive end of the scale due to the fact that if cost isn't an issue, stick a i7 in there instead!
If you're cost sensitive regarding the hardware, then you want the most efficient software possible to reduce the hardware requirements, and hence would not want the baggage of a general-purpose OS for a build-time-configured, static system.
There are plenty of hardware abstraction layers in the embedded world... (e.g. CMSIS), and familiarity is just a case of what you're used to, that argument could quite easily be turned on its head with a different set of people.
IP stacks also exist and are well ported to many RTOSs, (e.g. LwIP).
For example, a (very) typical embedded system that would be familiar to a lot of embedded devs would be: - Cortex-Mx (CMSIS for HAL) - FreeRTOS (scheduler) - LwIP (IP Stack) - Eclipse IDE.
There are always exceptions to this, e.g human-resources and development time issues change the direction you head in.
I spent a couple of years building a system to run in the "Sierra Wireless OpenAT" RTOS environment (like those ubiquitous SIM900 modules, but you could add your own code to them). It was pretty terrible and I'm glad they now offer modules that run Linux.
I agree that nommu Linux is a bit to heavy for most cases as it will always require external RAM and at that point you might as well consider a cheap Cortex-A chip (well, they are not so easy to source/aren't documented as good as e.g. a stm32) but I'd be more than happy if something like RIOT or ChibiOS got more traction.
but that baggage also includes such juicy things as a hardware abstraction layer with plenty of drivers, a network stack with 6LoWPAN and a familiar programming interface.