The graveyard of hundreds of abandoned open source game engines begs to disagree with you. I agree that it seems absurd, but you really can't underestimate the amount of man hours put into these things. Unreal has been under constant development for over 20 years.
It's the same as flight simulation. Why is there only one viable flight simulator (X-Plane) on the market? It's because that kind of coding is simply beyond 99% of developers, and it takes a huge well funded and focused team to make something state of the art.
I wouldn't cite open source codecs as a success story ...
A GUI toolkit is probably more on point and of the correct scale, and those also demonstrate that without significant backing from somewhere, progress doesn't get made at that level of complexity.
A commercial quality game engine is a huge project, and as such, requires a huge projects worth of time and effort to build. Same as any other huge software project (which all have countless abandoned open source attempts). Remember that most open source projects start of tiny (one developer is common, certainly very few start off with more than a handful) and work on it in their free time, while commercial teams like Unity have hundreds of peolpe working on it full time.
The reason they are abandoned is typicaally that 1) the developers, who are doing it in their spare time, don’t have the time anymore; or 2) they get frustrated at the amount of work/lack of contributors/lack of interest and stop working on it; or 3) they simply lose interest and work on something else.
Its not lack of ability, its lack of time/motivation/interest to keep working on it for many, many years.
- Understanding of modern CPUs / GPUs and the capabilities/shortcomings they have
- Understanding the behaviors and quirks of the OS
- Familiarity with multicore programming
- Skillful memory management (such as making your data structures less prone to cache invalidations)
- Mastery of a low-level language (C++, Rust, whatever)
- Familiarity with build systems (CMake, SCons, etc, ... or sometimes you might have to roll your own)
Already I could see the number of contributors who would know all this stuff being pretty low. (I've dabbled a bit in graphics and made a simple OpenGL scene graph renderer and a fluid simulator, but I still don't feel like I'm competent enough...)
An engine is comprised of multiple components (Graphics, Sound, Physics, etc...). So you wouldn't need to know all of it, but contributors would need to have a clear grasp on at least one. So for each component, you would need...
- Entity System: You need a universal way to handle game entities. Implementing it efficiently requires a good knowledge of computer systems, because you need to think carefully about how you store data in the memory.
- Job System: You need a system to execute various tasks (rendering, physics, game logic, ...) simultaneously on multiple processors, so that a single game frame can finish in less than 16 milliseconds. Implementing this would require some knowledge in concurrency as well as OS concepts.
- Resource Management: You need a way to store and keep track of the game's various data stored in files, such as images, models, sounds, game objects and their parameters, settings, etc. You need to implement efficient serialization and compression for this.
- Graphics: Mastery of a graphics API (OpenGL, DirectX, Vulkan), and understanding its various patterns and shortcomings. You also need knowledge in how rendering works (which also requires some math), and nowadays a PBRT renderer (https://www.pbrt.org/) is basically a minimum requirement for modern 3D game engines.
- Physics: Lots of math and physics (Linear Algebra, Differential equations, Mechanics, etc.), and also optimization skills like assembly and SIMD (speed is critical in this area!). Fortunately many people just use already-built middleware such as Havok or Bullet..
- Sound: This is actually the most neglected part in game engine development, and it is hard to find good resources on this.
I think the people who would have the knowledge to build a game engine would be already busy working on other stuff, such as working for an AAA gamedev company (which sometimes does a lot of crunch, so they might not have enough time for open-source stuff), or a proprietary game engine company like Unity or Unreal. But I still see some hope around the edges. From what I've seen, Handmade Hero (https://handmadehero.org/, which is a livestream of making a game engine from scratch, accompanied with various education materials) is the most impressive. It tries to teach those skills needed to make a game engine to interested programmers, which I think is a crucial first step in encouraging open-source gamedev.
Some examples: for entitiy systems, EnTT is amazing. For a job system you’ve got stuff like cpp-taskflow, for physics there’s bullet, for graphics there’s bgfx or full blown renderers. Sure, these won’t solve all of your needs, but they’re a good starting point that frees you from having to do a lot of stuff yourself.
You need to be damn near Johnathan Blow levels of skill and perseverance to create a decent game engine.
Maybe you wouldn't classify him as an "elite", but I sure would.
> There's probably less than a few thousand people on earth with the level of skill and expertise to build something at the scale of UE4.
That's really bunk. I mean "only elite can do it" it claim. Sure, it's not something you can create in one day, but it's not something that out of reach either, especially when people are collaborating on it.