If you can improve culling before rendering and physics, minimize the surroundings streamed in to your immediate vicinity, and optimize some of the heaviest materials and post effects, that goes a really long way.
Of course, there's an endless list of caveats depending on what your game specifically is, but even in an open world game where things happen off screen the above applies. You just have very simply LODs/imposters for faraway visuals and simplified logic running at lower frame rate for characters in other parts of the world.
Existing engines are great as long as you want your game to look and behave like every other game written in the same engine. It's gotten to a point where a careful observer can tell which engine a game is written in just by looking at it and feeling how much input lag and framerate variance it has.
I wonder if there's a DBMS out there somewhere that already does what you need? Seems like by the time you got to a scale where you would need custom stuff, you'd have money to redo it(Unless it's a foss thing meant to be big but not commercial, or to run on a server you're paying for yourself)