The downside is that PackageCompiler (or SnoopCompile, for that matter) can only pack into the sysimage what it sees. This is usually done by "recording" which methods get called during a session and putting the compiled native code for those into the sysimage. If it's not called during setup, you'll still have dynamic compilation (as for any code that hasn't ever run).

I’ve never tried it myself, but can’t think of any reason why it wouldn’t work in principle if you do want to go that route