Most mod-heavy games support that, going back to the classics like Unreal Tournament 1. Generally using Lua or some other embedding-oriented language, and minimal security audits. Anything else has a prohibitive workflow, and players won't exactly be auditing the mod-packages they run.

Lua is well-sandboxed for this, but I doubt devs are auditing the entire API they present to Lua.

And no, there are no security audits on engines like this. The only reason the mods havent installed a cryptolocker is because nobody has mad a really concerted effort to do so.