If your WAF is the edge of your datacentre, it will likely be ran by some sepperarte team with a generic ruleset, literally be MITMing all your connections, but more importantly it will be attackable via your company VPN or some other side channel like B2B connections generally, that you probably have no control over.
Your WAF, if you choose to have one, should sit directly in front of your app and have rules tailored to your app. Even those on VPNs, or in the datacentre , should be forced to traverse the apps WAF.
That said, the WAF logic is largely what you're going to do in your input validation layer, so unless you feel you need to double bag it try and to mitigate unknown framework or library issues, or are having another person write the rules, then you're just repeating work aimlessly.
When some dickhead manager buys an insecure http service they want on the web, you should probably quit. Failing that put a WAF in front of it.
I generally recommend having the WAF live in front of of apps, but with a small gap that allows your scanners to still access the app. It is important to still be identifying vulnerabilities that your WAF might block, because they aren't perfect.
Could you please elaborate on what some of the right things generally are?
Just curious, how this is covered from legal standpoint? Do you ask clients to sign some consent document to release you from liability from damage from hacking by automated tools?
I like to remind people that having someone on retainer that breaks your stuff with an automated tool is better than having some random guy on the internet break your site with the same tool. Either way it is eventually going to happen, but at least I will immediately notice and contact the right folks to help get it fixed.
Okay this sounds incredibly interesting. Are there any open source groups that do this "for free" sorta? Not asking to get free working, asking so I could possibly jump in and learn.
https://microcorruption.com/ is a quite a good one to get started with. (Because embedded systems are smaller and do less, they are easier to reason about, which makes it easier to see vulnerabilities.)
See also: https://www.reddit.com/r/securityCTF/
More broadly, the (with-permission) attackers are known as "the Red Team", and the concept is used against computer networks as well as in literal war-games by the US government.
From your profile it looks like you work for a big company. Ping some people in security and find out who runs your "Red Team" (if you have one). Make friends with them.
imho try to hack yourself first
There is nothing I love more than the "just throw {Cloudflare,Akamai,Fastly,etc} in front of it" crowd. If you are going to deploy a WAF [1] it _needs_ to be at the edge of your datacenter or cloud environment. No matter what fancy feature your provider wants to sell you to fix this, there is a way to bypass it.
* Make sure you are logging centrally, and logging the right things.
* Make sure someone is looking at the logs, it doesn't have to be a full time person, but it needs to be someones job.
* Depending on how big you are, hire people to break in on some regular basis. It could be anything from hiring a cheap contractor to run a bunch of automated tools, to full on 10 person adversarial teams. They will find some stuff for you to fix, but more importantly you will know whether the first two points are working based on what you see.
1. https://www.owasp.org/index.php/Web_Application_Firewall