i have yet to encounter something that shouldn't be expressed as a graph. three, babylon, ogl, blender, gltf, cad, games, they're all scene aligned. that doesn't seem to be a conflict since you still use shaders, physics, ecs and so on.
could you go more into detail what you mean when you say "anything non trivial"? is there a real example of something that would not be possible to create in, say, threejs?
Let's say you're building a game where you want a sphere to stick to whatever player you throw it at. How would you do that with a scene graph/OOP model? It'd be awkward, removing objects from one parent and adding them to another. Even more awkward if it's a complex object and you only want a part of that complex object to stick to the player. ECS + a constraint or physics system does a decent job (not perfect) of handling this in a relatively elegant and performant way.
I've used Three.js enough--built my portfolio[1] out of it, and then switched to Babylon when I realized how little I liked Three.js. For the record, I also dislike Babylon.
[1] https://tuckerconnelly.com