OpenInventor could have been it, but SGI had other plans for it, and no one at either ARB or Khronos actually cared about it.
Though i disagree with OpenInventor, it is too complex and takes too much upon itself, which few wanted - see Direct3D Retained Mode which despite being a much simpler API (both compared to D3DIM and OpenInventor) didn't see much use and in an uncharacteristic move by Microsoft (especially at the time) it was removed from Direct3D.
A better solution would have been something like GLUT, but with a few more utilities thrown in (like vector math stuff - OpenGL implementations already have the code anyway, why not expose it?). And GLUT was more popular than OpenInventor ever hoped to be despite not offering more than a few basic things.
The ones that don't use one, end up reimplementing their own scene graph anyway.
As for the rest, full with you.
OpenGL, Vulkan and Direct3D are at a level where they enable you to write your own engine, but not at the level where they provide your the engine themselves. Microsoft tried it with Direct3D RM and it didn't work and Sun also tried it with Java3D as the official way to do 3D in Java but also didn't catch on (it caught on more than D3DRM but that is mainly because there was no other official way - however its popularity paled in comparison to OpenGL bindings that appeared soon after and nowadays it has been reimplemented and lives on top of these bindings).
I have written a bit of Vulkan code (i wrote this[0] the day the spec came out, after banging on it for several hours - and i found the spec quite readable, at least from the side of someone who wants to use it... someone i know who worked on the implementation side has told me that it wasn't that great), but i find the API way too ugly and verbose for my taste.
I have considered writing a small OpenGL-like wrapper on top of it since i am concerned that OpenGL quality will deteriorate in the future (though that would break a TON of games, including the ultrapopular Minecraft), but for now things work fine.
[0] https://i.imgur.com/rd8Xk84.gif