https://www.cloudflare.com/application-services/products/waf...

<Receives solution>

Nah I’m not opening that link, I got it

I wish this was a joke, but just last month I spent literally hours arguing with multiple people -- on shore -- that that kind of query rewrite/rejection approach was never going to work properly, and only properly parameterised queries were correct.

Nope.

Fix after fix, then fixes for the fixes, then workarounds for the glitches, and then... on and on.

It was incredible to me that in 2023, supposedly senior technical team leads would have heated arguments rejecting parameterised queries and favouring regex WAF instead.

Source: After so many years of dealing with bad technical decisions became a TL myself :)

...what do you mean by "rewrite/rejection"?

If rewrite means "escaping strings using the database function designed for that purpose", then that approach works just fine. It's not comparable to rejection at all.

If they were making their own version, then the underlying problem is that they were making their own version. Parameterised queries are lovely but they are not the only option.

That means that when a user called Bob O'Neill enters their name, instead of returning a HTTP/500 error, the database stores Bob O''Neill.[1]

Then when the user goes to edit their form, they will see O''Neill. Okay, oops, that's a mistake, let's just replace all double single quotes with a single quote when outputting HTML! Now it'll say O'Neil correctly!

Of course, if you enter some bad text with double single quotes via some other mechanism such as a CSV upload, there's a decent chance that'll it'll be incorrectly stripped. Perhaps in some mid-tier API, which will then interpret it as a single quote, resulting in an injection vulnerability (or data corruption) again.

That can be fixed with "mere" man months of effort instead of the minutes it would have taken to just use the parameterised queries like God intended.

Now that that nightmare is over once and for all... what to do about % symbols screwing up LIKE searches? I dunno, that's complicated, so let's just replace all...

... rinse, repeat, ad infinitum.

[1] Oh, oh, you assumed that the query engine would replace '' with ' and the database would store the correct text? Hah-haaa.... you assumed that this "fix" was applied only once! What's fun about band-aids is that they're so easy to accidentally layer three or four deep without even realising. More band-aids == more safe, am I right?

YOU HAVE BEEN BLOCKED FOR MALICIOUS ACTIVITY surely has to be good for business. Not that most would know, considering the trackers won’t load when this happens.

Works. Any sql thats actually blocked?

In this case, it looks like some comments are giving good advice about the tradeoff and how to fix it, so I have to agree with you.

An alternative is to keep all of your CDN assets on a CDN bucket on its own hostname, with your main secret-containing business apps on your own servers, but it costs a lot to manage this level of separation and the payoff is only protection against the theoretical attack of "NSA can't attack our users/spy on them". If the NSA ever did do this on a large enough scale or to target a particularly notable person, it's very unlikely it would be kept a secret for long, and the end-business that used Cloudflare et al. wouldn't be implicated whatsoever since every business uses one of the big CDN providers.

The only weak link now is Cloudflare, which is still "less secure than a direct connection" (with respect to government spying, bugs[0], hackers, etc) but the threat level is drastically reduced.

0: https://blog.ryankearney.com/2013/01/comcast-caught-intercep...

1: https://news.ycombinator.com/item?id=13766339

I'm guessing it's a sufficient condition, bit not a necessary one. I.e, a could be using Cloudflare's WAF with a SSL cert issued by somebody else.

0: https://developers.cloudflare.com/ssl/edge-certificates/adva...

https://news.ycombinator.com/cdn-cgi/trace

Every Cloudflare site will respond to this URI.

That said, the choice is yours whether or not to use sites that utilize such untrustworthy MITM providers, like Cloudflare. There are even browser plugins that can automatically block connections to such untrustworthy entities.

This isn't an endorsement, and you should always review the source code of any browser extensions you're utilizing due to the risks extensions themselves can pose, but I personally use one called Cloud Firewall and it works great. (https://addons.mozilla.org/en-US/firefox/addon/cloud-firewal...)

There aren't obvious signs up front that a site is using cloudflare. Failure to spend time investigating is not "freely choosing it".

You're joking, right?

It takes 2 seconds to click the padlock in your browser, click through once more, and see "Verified by: Cloudflare, Inc". You don't even need to view the certificate.

If 2 seconds and 2 clicks is too much time and effort, it's obviously not actually that important to the user in question.

https://developers.cloudflare.com/ssl/edge-certificates/cust...

It's not always that simple.

https://i.imgur.com/TO2Tfk3.png

https://i.imgur.com/jVW5db4.png

This is also how they insert extra headers in both the request and response.

F5 Networks, my former employer, sells something similar, but it's a box (or virtual appliance) you put in your own data centers somewhere that dead-ends the connection instead.

It's entirely possible to have a proper SSL connection to a bogus hostname, that is showing the correct website and even interacts correctly.

Bogus MITM decrypts the traffic, logs it, then forwards the traffic once again encrypted to the destination server. Then does the reverse for the resonse.

"Look for the padlock" is only useful if the actual hostname is correct in the browser.

If I hosted news.ycombnator.com using this and you didn't notice that I could be proxying just like that. It's possible cloudflare has protections against this in place but doesn't every website on earth?

Look at the damned hostname people.

    cat testfile.txt | nc 192.168.2.100 1234

> stealing from users

What a weird definition of stealing.

And it's as much an issue with your browser if hitting back doesn't return the text. There are extensions to improve that behavior.

But what I find really interesting is that you seem to think being mad about an issue is a reason to break a completely unrelated rule?

Your time is apparently free, given your bafflement at how stealing it is stealing. I don’t know what you’re on about with “browser issues,” because that’s not it. If you can’t post, the button to raise the text-entry UI should be DISABLED. Anyone on this site should understand and expect that. This has been an egregious abuse of users for years, and there’s no excuse for it.

As far as what “rules” you think I’m breaking (which you didn’t manage to specify) and being “mad…” I’m not the one flagging and burying other users’ posts. If I annoy pompous, self-important pricks that’s a win. One of them has gone through and downvoted all my posts now, proving my point: HN is going the way of Reddit. Flag away.

And I still don't think stealing is the right word for that kind of technical issue, especially when it's still half your browser's fault.

> which you didn’t manage to specify

Why would I need to specify something you brought up? "OH NOES comparing HN to Reddit violates "policy.""

> and being “mad…”

Is "aggressive griping about" better? People usually simplify that to "mad about".

> One of them has gone through and downvoted all my posts now

Almost every post you made inside that 24 hour downvote window deserves it, so depending on how literal that "all" is, they're probably helping and not a bad actor.

Minimal test, if I try to edit this post removing the asterisk, I get the "banned" page

  nc* 192.168.2.100

Any idea whether that's just an overloaded application server, or something Cloudflare is doing?

The holy grail of NIH :)

0: https://news.ycombinator.com/item?id=38939668

The main YCombinator site might have been using Cloudflare seen years, and now HN might have been added to the same account to protect from a DOS attack.

So might be the same cert?

It's very easy to disable it completely via Cloudflare settings. Using cloudflare doesn't require you to use all of its features, and almost every feature can be turned off.

Is there a reason it needs to be visible whilst performing checks? or is it just security theatre?

WAF in general is security theatre. If your operation genuinely benefits from one, I dread for what's sleeping underneath. Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.

When it's needed, it's needed, but it amazes me how many people feel they need big brother protection for their personal blog and nextcloud

Unless your client sends a string which matches one of the WAF patterns the site will work fine. It only blocks individual requests.

Now the problem here is that you probably shouldn't enable the WAF without having it in log only mode for a while if you are operating a site which let's users submit arbitrary text input. Of course it's going to match... You'll have to adjust the configuration.

I’ve had to recompress zip files with a higher compression setting to get around whatever string was triggering it.

edit: my memory is crap. it was a single machine, but the codebase was written in a custom experimental language that i think was a lisp derivative. (which would make sense!). the source was online at some time, can't find it now.

[0] https://news.ycombinator.com/item?id=38310213

[1] https://arclanguage.github.io/

[2] https://news.ycombinator.com/item?id=26473226

Edit: Seems to work also for at least some other Cloudflare sites. Interestingly HN is served from Stockholm (behind a sea cable) while others are served from Helsinki (should be closer). Not enough hackers here in Finland?

Edit 2: Works also on sites where Turnstyle keeps me out.

    news.ycombinator.com. 3710 IN CNAME news.ycombinator.com.cdn.cloudflare.net.
    news.ycombinator.com.cdn.cloudflare.net. 3710 IN A 104.22.6.236
    news.ycombinator.com.cdn.cloudflare.net. 3710 IN A 172.67.5.232
    news.ycombinator.com.cdn.cloudflare.net. 3710 IN A 104.22.7.236

  ../ ../ ../ etc/ passwd

Also: https://www.nslookup.io/domains/news.ycombinator.com/dns-rec... / https://archive.is/R9BE8

   nc -l -p 1234 -q 1 > testfile.txt < /dev/null

   alias foobar=nc
   cat testfile.txt | foobar 192.168.2.100 1234

This is also fine, but blocked if you change the slash to a dot:

   nc 192/168.2.100 1234

   nc \
   192.168.2.100 1234

  socat - TCP4:192.168.2.100:1234

  socat TCP4-LISTEN:1234,fork,reuseaddr -

I think it's like a lot of things in computer security, in that system owners just don't want to be the slowest gazelle in the herd. If an attacker is mass-exploiting some new remote vulnerability, then maybe the WAF means that you're one of the lucky ones who doesn't get hit. And yes, that's a very big maybe there.

WAFs don't do much to prevent targeted attacks except to require the attacker to craft a WAF bypass. As you've shown.

(I'm exaggerating here, but only a bit)

The key is that they should mitigate specific vulnerabilities and ideally once the proper fix is deployed the rules are then removed from the WAF.

WAFs have near-zero value for things like trying to detect shell-code or generic SQL injections as they just turn into fuzzy bug injectors. Any real attacker will very quickly find a way to structure their exploit to avoid the heuristics.

WAFs also have minimal value for custom software. Because even if you use it to deploy a quick block for an exploit unless you are just blocking a whole endpoint the attacker will also likely find a way to work around it.

I can add to the list of attack vectors a case where the WAF introduced a reflected cross-site scripting vulnerability. The site it was supposedly protecting was blank, i.e. it just returned a 404 error or something. But just by appending a URL parameter with JS in it, the WAF would trigger and reflect the code. So I was able to build an outlook web app lookalike for phishing on a site with the domain of the company.

1. WAFs require entire requests to be buffered in order to be scanned before the server sees them. This can require lots of RAM.

2. WAFs scan requests with all sorts of hacky rules, which takes gobs of CPU time.

3. The hacky rules look for programming language syntax, for which the attackers can easily find alternative expressions to get around the rules.

4. ... yet, WAFs have high false positive rates.

5. All that kludgly processing is a security weakness. WAFs tend to be closed source behemoths written in low-level languages.

The question what is the alternative and the suggested alternative is that everyone become perfect security expert. It is even less likely to succeed than creating security-aimed software by professionals.

Consider what is the proportion of sites that are created by people who knows zero about security. Wordpress is like on 40%+ web sites (ridiculous).

WAFs unlikely to prevent targeted attacks, they don't have to to be useful. In practice, simple measures can prevent many common attacks.

I have to disagree here. You are making assumptions that every developer in an org will always do the correct thing and deploy code that won't be exploitable to SQL injections, XSS, file inclusion, etc... That's just not the case. I'm all for doing the correct thing, and not just performing security theater, but WAFs do offer some protection. You need multiple layers of security covering the holes that may left in other layers. And a WAF can be one of those layers of protection.

Yes, they can be trivial to bypass. And yes, they don't block everything. But also, yes, they can be very useful in some situations.

My employer's security team maintains a WAF, and while it may be frustrating at times (like when anti-directory-traversal rules broke page names with '...' in them) I mostly prefer that they continue to do so for two big reasons: script kiddies and botnets.

It doesn't matter that a bypass is trivial when in practice your attacker won't mutate their attack -- if the attacker was more sophisticated, the defence could be too, but if the attack is dumb then there's no point in a sophisticated defence.

Botnets mean that purely reputation-based defence is insufficient. The best defence to a distributed attack is one that's really cheap to evaluate. If all an attacker ever tries is to hit our homepage with a fixed user agent string, then all we need to do is block that UA from hitting our home page. A simple WAF entry is sufficient to block that particular attacker.

This precise example is indeed poorly-applied, as the system is intended to receive arbitrary text of arbitrary technical complexity. But I wouldn't mind the rule being applied to my team's endpoints, as we can be confident that anyone sending shell has malicious intent regardless of whether there's any chance that my services would try to execute the code (they won't).

So long as it is possible to bring down services without any effort, skiddies will keep trying to do that. And so long as we've people trying dumb attacks in infrastructure, dumb defences can have a worthwhile effect. And if the dumb defences start catching stuff they're not supposed to catch, like the example with '...', they're dumb enough that we can understand why they're doing that and if we can safely turn them off.

`nc 192.168.2.100`

Additionally cloudflare doesn't know what is safe for a given site, so it has to be a little conservative. The sites that can handle malicious input, or are tech sites that expect things that are SQL or commands that may contain directory traversal, these are in the minority.

Essentially these are false positives, which are typically viewed as more acceptable than false negatives as those would allow attacks through.

These things are configurable by the site owners, but the issue here is that the site owners are not shown the code of the rules, so have to guess from the names and descriptions whether something is safe to disable, meaning everyone just leaves everything enabled. Usually reporting this to a site owner with the cloudflare trace id is sufficient to enable the site owner to disable a rule that is causing false positives, as the site owner can use the cloudflare dashboard to search the trace id.

I do not work there any longer (left 3 years ago), but did write significant parts of the firewall and also manage the firewall, WAF, and DDoS protection teams.

    nc 192.168.1.100 8000

Well, it's a client that has no Javascript engine.

There is no need to use Javascript. This works for me just fine:

https://i.imgur.com/YtepoDbh.jpg

Just replace "i.imgur.com" with one of the Rimgo instances:

https://rimgo.codeberg.page/

(The most popular ones are usually rate-limited though)