Not the person you're replying to but opposite of open source is closed source, and isn't that basically a black box to you since you don't know what it's doing?

Is there something more subtle I'm missing?

Yes. Microsoft offers source access to Windows. IBM and Oracle will rent you people who know the details of their software. None of those companies' offerings are particularly ‘black box’-y, in spite of being very closed source.

‘Open source’ is more about the development model (and freedoms) than about the nature of ‘knowing what the software is doing’.

Heck, I could argue that Linux is a black box to most people who aren't well-versed in kernel development. OpenSSL is notoriously difficult to understand. Sometimes huge bugs look relatively innocent¹ even with people looking at the code.

¹ e.g. https://lwn.net/Articles/341773/

At the level of security where you need to be paranoid about the operating system developers, how do you know the source code Microsoft shows you, corresponds to the Windows binaries running on your machines? (Let alone to the sum and total of all binary patches applied thereto?)

A source license probably also gets you the build system, or if not maybe you could throw more money at them.

But reproducible builds are a hard problem, so… honestly, I can't answer that.

And that's what I really mean by 'black boxes'. Yes, you can buy access to, e.g., Windows source, but that doesn't mean that what you review and what is running on your systems are the same thing. And I can't even speak to all the third-party bits that get compiled in to an instance of Windows. It's a really, really hard problem.