https://developer.apple.com/documentation/xcode/notarizing_m...
> For example, if a plug-in employs deep integration with the host executable via C function pointer overrides, or uses a JavaScript engine for custom workflows, the host executable must declare the Allow Unsigned Executable Memory Entitlement or Allow Execution of JIT-compiled Code Entitlement, respectively. In some cases, a plug-in fails to even load if the host executable lacks the proper entitlement.
This is the best documentation I could find with a quick Google search. But basically, if you can use C function pointers, you can call anything you want and there is no way that an automated scan can detect it with the right level of indirection.
I also couldn’t find anything from the official documentation about not being able to code sign anything that uses private APIs.
But even simpler for apps that don’t actually get manually reviewed, it’s really easy to bypass.
https://medium.com/swlh/calling-ios-and-macos-hidden-api-in-...
Even if they did do something like scan for text strings in PerformSelector, that’s also easy to obfuscate.
> The Notary service will also perform some additional checks on the application. These include security checks that verify the application is doing what it indicates as well as the check for private API usage, similar to Mac App Store apps.
[1] https://www.waynedixon.com/2019/05/16/new-notarization-requi...