For me the definition of 'layers' was having to juggle dozens of interacting state flags and mutually exclusive vendor-specific extensions just to draw a dual-textured triangle or render to a render target.

COM really wasn't that much of a hassle in comparison. A couple smart pointer templates and you're off to the races. I can see how a C developer would really resent it though - nothing but a pain compared to GL's regular C, 'everything is void*' api.