I mean, yeah. If you try to use a framework intended for games, it won't be a great fit for other use cases. Look into Open Scene Graph if you want a higher level API that isn't as opinionated towards games.

Vulkan isn't a higher level API though, which might be why you and the other guy are kind of talking past each other.