I think your logic stands when thinking about individuals or private organisations.

But when thinking about state actors they almost have unlimited money/resources thus even if adding a backdoor is a bit harder on open source having unlimited money/resources makes it irrelevant.

Where “unlimited” = to the size of the open source organisation that maintains a bit of software.