Consider a client that emits \n instead of \r\n. How do you handle it? Liberally? OK, treat 'em like CRLFs. Now you read \n\n. Everything after that is content, right?
Oops, you're now ignoring headers, potentially security-sensitive ones.
I've run into this exact bug in production, leading to a security problem. The client, proxy, and endpoints had different ways of handling CRLF. Some would treat \n\n as the end of headers, some not. Exploiting this, clients could route requests through the proxy and add special headers that only the proxy should have been able to add (like X-Client-IP).
Apart from this, the whole "robustness principle" just leads to a bunch of guessing and even more incompatible implementations. See HTML as another example mess.
Well, that's the rub, right? How do you know how strict you're being if your tools accept things liberally? If anything, the lesson here is to test with the strictest possible tools.
> just as the Cowboy was not liberal in accepting
And this is hard too, because on what dimensions should you be liberal? How do you decide what the "real" set of inputs you're going to accept?
And that leads to my real issue with the principle: what should you, as the liberal accepter, do in those cases? Here it's easy enough to guess what the behavior should be with the extra space (just accept the damn request), but in general it's not -- you're creating implementation-specific behavior; what happens when you accept undefined or incorrect inputs will vary from implementation to implementation, creating a nightmare of uncertainty for people sending you stuff. Of course, you can always say, "they should send stricter stuff!" but then what's really the point of accepting inputs liberally?
So different software will necessarily do it differently. For all software to be doing it the same, there would realistically need to be some specified standard on how to do it, and then we're no longer talking about 'be liberal in what you accept', but just 'accept exactly what the standards say.'
Of course, in this case the client software was not being 'strict in what you issue' -- I am not challenging that part, of course you should _always_ issue exactly correct according to standard requests or other protocol communications. But there will inevitably be bugs, bugs happen.
"Be liberal in what you accept" makes it harder to find those bugs, and leaves them waiting to surprise you when the (non-standard) level of "liberalness" on the receiving end changes, which it inevitably will because it was not according to standard in the first place.
I think the HTML/JS/CSS web provides another good example of the dangers of 'be liberal in what you accept', very similarly -- you may think your web page is 'correct' because one or more browsers render it correctly while being 'liberal', and not realize it's in fact buggy and will not render correctly on on or more other past, present, or future browsers. This example has been commented upon by others, and I think has led to a move away from 'be liberal in what you accept' in web user agents. http://books.google.com/books?id=5WXp4j4eV4UC&pg=PA136&lpg=P...
Be strict in what you issue (duh!), be liberal in what you accept - but both emit strong warnings when the input isn't strict, and have a strict mode.
If everyone can be strict in what's sent, then the problem is solved. But since that won't happen, even on accident, the only solution is to be harsh on receiving input and hope things fail early in the dev cycle.
Also, text-based protocols are especially prone to this poor handling, A: because spec writers (like HTTP's) go moronically overboard, being all creative (line folding? comments in HTTP headers? FFS!) and B: because text is so easy, everyone just figures anything goes and pays less attention.
I think the fallacy is to assume that once stuff works in production, only your changes can trigger a bug. There's way too much software involved in a standard webserver stack to assume anything about it. Any patch, any update to software or devices not under your control has the potential to break your stack. The thing the OP did was the right thing: Monitor, monitor, monitor.