Look into oom_kill_allocating_task. There's likely a better solution such as an earlier OOM, but killing the wrong process seems better than locking the system.