I spent some time in the trenches on this, and concluded what is needed is some sort of agreed module/integration mechanism and an ecosystem of open and proprietary modules. (Quite similar to how many large game devs operate already).

No “open engine” will ever be enough, by definition, as the games people will pay for will be those which were hard to make, and tech advantage is one well proven way to do that.

The tech industry is beginning to learn that barriers to entry, such as those Nintendo created in the 1980s or existed in the early days of the App Store, actually helped the market function to create much better results. Apparently Atari will be repeated again and again.