Meanwhile we have been compiling the same C and above code for mips, arm, power, x86, and riscv for decades in portable ways while still getting performant code.
Now to be fair GPUs have a legacy of slowly becoming generic processors whereas 10+ years ago they were largely fixed function hardware. Writing generic code for a GPU today is totally sensible because it supports most arithmetic, primitive types, branching, etc.
But there is nothing stopping you from optimizing your compiler for glsl any more than optimizing your platform specific custom vendor compiler for C.
You still write performance-critical code using assembly language. C compiler is not ideal.
The problem here is that abstraction comes at a cost, and that cost is so great that we need even crazier abstractions to be able to work around it... and now the suggestion is, as you say, work with a whole engine. Ah, now I can draw a pixel on screen! I see it as a lot of wasted potential (even if maybe not that much wasted commercial potential).
Probably the best thing you could do at this point is to use game engine like Unity or UE as an abstraction.