It just comes down to incentives. Practically every living human being stood to gain from the existence of a high quality general purpose FOSS operating system, and so Linux naturally came about.

The people who would benefit from a AAA open source game engine are... closed source AAA game development studios. As such there's really no incentive for some physics whiz to spend their extremely valuable time toiling away on something that is only going to be used for someone else's profit. The intersection of people who build game engines, and people who make games with those engines, is extremely small.

There's really no such thing as open-source game development except for hobbyists, because that would be literally giving your product away. Games are a hit based, one-and-done type product much more akin to movies than other software.