except the easy common things are very slow, and the optimizations which are basically required become a careful dance of understanding the underlying hardware interface anyway (without any real documentation) and figuring out how to map that onto the high-level interface.
OpenGL is exactly what you said, "an API where the common things are easy, and the super powerful low-level optimizations are optional."