I'm talking about guardrails that prevent finding exploits, which is only peripherally related to writing secure code.

This benchmark is about finding security bugs, not writing secure code. I don't believe the models have guardrails that prevent writing safe code, but they're also not intelligent and have a bunch of insecure code in their training data, so they definitely write insecure code sometimes.