No, we definitely should punish bureaucrats and enforcers who act negligently. If someone in a position of authority flagrantly fails to do his job and it directly harms someone he should be held accountable. That would provide a strong incentive for future actors to take their responsibilities seriously.

If an engineer signs off on an obviously faulty building plan and people die as a result we hold him accountable. This is no different.