If I require 5 ports to be hit in sequence, and blacklist IPs that hit unknown ports, it is extremely unlikely you will ever connect. Now if someone on my local network, at my ISP, or at my hosting provider sniffs my traffic to determine a static knocking sequence... good for them. They're the one unauthorized person who can connect to sshd, without a valid ssh key to authenticate with.
It's a reality that most businesses are not going to invest in setting up a network that cannot be accessed from the internet at large. For such setups, a little bit of obscurity via something like port knocking to prevent every single port scanner in existence from discovering your sshd server must be better than nothing at all.
so what happens when someone hits an unknown port on your system from every IP on the internet?
That, and having out-of-band access to the server always helps. ;)
They make the valid criticism of the first case while you argue the second and somehow the arguments miss in the middle.