Technically, they could. It would require someone who hasn't actually licensed the SDK, and so aren't subjected to an NDA, to reverse-engineer things and produce their own implementation under an open source license.
Certainly would be an enormous project, but it is well within the realm of the possible. It's been done with complex systems before.
Yes, I know it's possible to reverse engineer the consoles, but that doesn't make it a viable alternative for the games industry.
It makes it legally viable, in that it would allow the production of an SDK that isn't restricted by any NDA, and therefore could be incorporated into opens source projects.
- The platform holders can and will ban software built without the official SDK
- You'd get delayed access to patches, which might break compatibility with new OS versions
- You're relying on someone being able to reverse engineer new versions for the life of the console. This is a rare skill.
It's like saying you can write your own C compiler. Yeah you can, but you wouldn't use it in production at your job.
None of your points appear insurmountable or showstoppers to me, to be honest, but they would require approaches to mitigate them.
Nintendo Switch and Playstation and titles from Sony and Nintendo incorporate BSD-licensed open source code, so it is obvious that “open source is banned” is not true. It’s only GPL and other viral licenses that lawyers argue is too risky, because it might require disclosing proprietary source when linked.
Same goes for Apple App Store as well
https://news.ycombinator.com/item?id=12827624