I've been gradually removing cloudflare based CDNs from services I develop and control because I don't want my users being arbitrarily discriminated against.

There was a good article posted on HN recently titled "The ideal level of fraud is non-zero" which I think is highly relevant here... In essence any mechanism employed to prevent illegitimate use comes with a negative cost to legitimate users, if that cost is too high it defeats the purpose. i.e what's the point in a website that is completely immune to a botnet and also cannot be accessed by anyone else? unplugging the ethernet cable also effectively protects against botnets. More subtly the cost of outright rejecting some legitimate users is usually not worth the savings of rejecting 100% of illegitimate ones. I think Cloudflare's service has it the wrong way around: it currently accept blocking legitimate users far too easily, that is not an acceptable cost; whereas you should be letting a higher level of bots through to avoid pissing off legitimate users - if it's not obviously a DDoS, it's probably worth the bandwidth cost.

Consider the bigger picture, if you save a slither of a penny by blocking a bot, but also end up blocking or seriously inconveniencing 10 real users... is it worth it.

The space is in need of solid competitors to break the stranglehold they have on the internet. Whether it's the right combination of services, documentation, etc.

But Tor is an enormous source of abusive traffic and if I don't filter it, then that's harmful to site owners. I'm being forced to choose between the needs of people that I know, work with, and depend on financially, and the needs of people in countries with issues that are far outside my ability to resolve. It's not a hard decision.

between 2015 and ~2020, my home ISP was blessed with every recaptcha being 3 rounds of slow fade-in bullshit. I have also seen infuriating gaslighting of "please try again" after certainly correct solutions, as well as 5+ rounds followed by a notification that my network is entirely blocked.

I've developed a reflex to Ctrl+W upon seeing it, unless that is absolutely vital for me to get past it - which is exceedingly rare.

if I had a genie lamp, I'd waste one of my 3 wishes to do terrible things to the people responsible for that shit.

Tor users do not have any special properties over clear-net users besides low accountability for their IP space. There are other ways to acquire this type of setup that don't involve broadcasting a public list of known exit nodes as an act of good faith. Any sophisticated attacker will be able to easily get ahold of the IP space and bandwidth they need to do their work, whether it's through a botnet or simply because they operate out of some less-accountable country like China or Russia.

IP filtering: now you have two problems!

/s, obviously, I hope.

Blocking Tor isn't a security measure, it's a nuisance reduction measure.

You kid, but this is completely true Email is simply an incredibly flawed, outdated and broken system, especially when used without PGP. Phishing is a massive problem, and it has only continued to grow in scale because spam, uh... finds a way. At the same time, spam filters regularly create false positives, making email an unreliable transport (leading "oops, it got lost in my spam folder").

>Blocking Tor isn't a security measure, it's a nuisance reduction measure.

You should block all IP space, this will reduce nuisances by 100%. In fact, this will save you from having to consider any real security practices or do your job properly.

Tor's advocates in this thread keep trying to argue it from ideology, as though anybody's obligated to deal with Tor traffic on principle alone, and not one of them so far has tried to argue that Tor is not 90+% bots and garbage. Funny, that.

I have Tor installed, figured it would be worth adding my boring browsing to the mix sometimes, but since most sites I try to load block Tor exits, Tor browser now sits unused.

On the other hand, if I woke up tomorrow deciding to start a bot farm or whatever other malicious thing, or course I'd be interested in hiding through Tor and might try it again (don't worry, I won't wake up that way).

So even if a hypothetical 100% of global internet users really wanted to do all their browsing through Tor, they might all reach the same conclusion as me that too many sites are blocked and therefore leave Tor to mostly bad traffic. Of course it's nowhere near 100%, but hopefully you see my point that the sites blocking Tor IPs (and I absolutely appreciate why) can become a self-fulfilling prophecy - and I'm not sure how you'd get out of that loop?

https://www.linode.com/community/questions/22305/entire-ip-r...

Depends on what you imply under 'hard'.

As a IaaS provider I endured alk the hurdles about that and ten years later - I don't care, at least not until my outbound bill is bigger than usual.

Like some of the clients are on CentOS6, on a public facing machines.

What kind of abusive traffic is coming through Tor and why do they do it?

But the signup spam was a headache. I didn't want to just blackhole Tor traffic, and tried to reduce the abuse with other tools, including some custom stuff. The final straw was a customer's small business site that had a MailChimp or Constant Contact signup form. Those vendors want you to embed their code by default to render the form, so you have less control over the form itself. There were workarounds, but they all sucked.

Tor bots would sign up email addresses through this newsletter form, and then I'd have to go through and manually scrub them before newsletters went out, or the service would penalize my client for too many bounces/unsubscribes/complaints. Very nearly 100% of the abuse on that particular form came from Tor IPs.

I do not want to spend my limited time on this Earth manually sorting out bots from humans because of one particular network. Blackholing Tor made that problem disappear immediately.

VPNs are dime-a-dozen now, cheap VPSs are available from lots of vendors, there's Wireguard, there's ssh, a clever person could even set up Apache or nginx as a forward proxy with ssl from LetsEncrypt. Tor is well over 90% abusive traffic (https://blog.cloudflare.com/the-trouble-with-tor/). This is a Tor problem, not a me problem. There are better alternatives available.

Solution: Require sign-ups by email, so the end account must actively send your mailserver a registration message. This also turns an open-loop control system into a closed loop control system, which is inherently easier to secure / keep safe.

It's also easy to send "from" an addresses that passes SPIF/DKIM but bounces inbound mail -- not sure what reason someone would have for this other than hurting the service reputation or acting as a DoS of sorts, but it can be done.

Proper DMARC configuration is table stakes to send e-mail, which makes that anything but trivial.

Then put the form behind your monopolistic internet gatekeeper. There's no reason for a GET to redirect to a sysiphean captcha treadmill.

> . Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious. That doesn’t mean they are visiting controversial content, but instead that they are automated requests designed to harm our customers. A large percentage of the comment spam, vulnerability scanning, ad click fraud, content scraping, and login scanning comes via the Tor network. To give you some sense, based on data from Project Honey Pot, 18% of global email spam, or approximately 6.5 trillion unwanted messages per year, begin with an automated bot harvesting email addresses via the Tor network.

Using tor hides your IP address from the website and makes switching exit nodes very straightforward, so you can run your account take over script in peace.

So yes, you can switch exits easily, but effectively your switching from one known bad IP to another bad IP.

I wonder if that's such a bad thing. Tor is safer when the traffic never leaves the network. In the ideal world, everything that matters would be inside the Tor network instead of being merely accessible through it.

I did tell Amazon about it, but it fell for deaf ears. The ban lasts for about a week and the internet is mostly unusable in that period

> The past few weeks I've been getting tons of redirects to verify my humanity before being allowed to view a webpage. Usually I just have to click the box that says human, not find all the ladders in a photo. SoFi is doing it every single time I log in. Petco, too, along with others who are more sporadic. This is happening with and without uBlock on. Same browser I've always used. ...

SoFi and Petco both use Cloudflare. I do exactly zero web crawling / scraping / abusive anything from my home connection.

I'm noticing a recent increase in volume of complaints about Cloudflare's human verification filter. I'm starting to wonder if they touched a dial.

I had already started pulling some infra back from Cloudflare after their last appearance in the tech news cycle. Now I've got an additional reason to continue doing that.

That you know about. Your house mates share the internet connection.

I’m guessing you have WiFi, so you may have unintended guests.

You probably have lots of devices, one of which may be infected.

Your ISP may have issued you a different IP which may have a negative reputation score.

You could be using a malware infected browser or browser extension.

There are lots of variables. You haven’t isolated all of the ones in your control, so assuming CloudFlare is the only possible cause isn’t rational.

Scaling to infinity isn't a right, it is a privilege. Any company that builds this sort of no-human-decision systems are abusing that privilege and hoping that anyone who suffers wrongly under their systems doesn't have enough voice (google seems to be the worst for this, though cloudflare seems set to follow).

This is actually the least likely these days... the no.1 cause would be CGNAT, the vast majority of residential endpoints share an IPv4 address with a huge number of users, mobile networks are even worse... that's before we even get to IP recycling for dynamic IPs which happens at high frequency with mobile networks again, so you will inevitably get affected eventually.

This is why it's a bad idea to block IPs outright, because today one IP address never equates to a single individual or the same set of individuals over time. The other problem with blocking IP addresses based on abuse is considering them equal in user weight, yet one IP might have 2 users, another might have 10000 users - Blocking a TOR exit node is a good extreme example of this... people think of it as an effective defence because of the concentration of abuse on that single IP address, but they fail to consider the concentration of users behind that IP address - TOR exit nodes probably are a slightly higher source of abuse per user, but not any where as high as per IP - if you measure abuse per IP you are more likely looking a rough picture of users per IP for highly NATed IPs.

What triggered your reaction? That they terminated a customer with zero notice?

Essentially, for a broad class of web-based businesses, they have made themselves gatekeepers. I'm sure they'll find a profitable use for this position. Charging outright would look bad, but investing in businesses that just happen to not run into Cloudflare-based trouble, but whose competitors do...

I'd guess the intent is unlikely to be anti-competitive or monopolistic, just over-aggressive. However regardless of intent their position does cause an absence of market forces to put pressure on fixing such issues - Similar to how it's become acceptable to have downtime when it's on AWS, because "everyone is affected".

As with any power and control you delegate to any entity, only time and good behavior will earn them your trust. That's theoretically what companies are competing for, your loyalty.

I encourage those of you attempting to block Cloudflare to try and host your own website for a bit. Make sure you don't do it on a metered/paid connection. I know one eCommerce site with 1,300 employees that went bankrupt overnight thanks to the AWS bill (and lack of options to get back online, this was prior to companies such as CF). Bankruptcy as in the company filed for bankruptcy and no longer exists. They were profitable for a decade prior. One DDoS attack...

Also make sure you don't have a democratic opinion if you are in the US, like a 50 person manufacturing company. They were shut down completely thanks to saying a single wrong thing about Republicans. CF existed there, but they weren't aware thanks to not having IT folks. They were a non profit.

CF may be evil to some, but there is a reason they exist. I use CF. I don't like throwing money at them every month, however, many of my websites have also been attacked, usually via competitors. We can either deanonymize the internet or allow companies like CF to exist. There is really no other way.

Being milked dry by a single DDoS is a hosting issue in my opinion, there should be sensible limits in place, AWS is notorious for making it very hard to understand and control this...

Even if you disagree and consider it a problem that must be solved with a separate DDoS protection service, this is not what I am talking about, I think it's a good idea - if there is a clear ongoing targeted DDoS attack, that system needs to engage and do it's best to try to filter through only legitimate users (which is the only point in time it makes sense to potentially block regular users - because the alternative is that no one can access the site).

The problem is this is not how cloudflare's protection operates, there is no throughput trigger, it's always on, it attempts to block bots at all times and has a very high false positive rate.

Had they just called AWS and explained the situation, they would likely still be in business.

I keep a backup DDoS mitigation service for my entire network that costs me less than $200/mo to mitigate up to 100 Gbps.

It comes from the Cloudflare blog. https://blog.cloudflare.com/cleaning-up-bad-bots/

There’s a support page about it too. https://developers.cloudflare.com/bots/get-started/free/

Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.

PS: I love all the innovation and engineering stuff you guys regularly share on the Cloudflare blog. It’s [almost] always an interesting read. Even though I’m no fan of the massive centralization your company has caused.

I don’t understand why you saw one minute block screens. That’s not right. Should be seconds.

I’m talking with the team about your other points.

This is the most serious problem with all of the major companies these days. Cloudflare, Google, Apple, etc. When you get on their "bad side", you're just screwed. You'll never even know what got them mad at you, and there's nothing you can do to recover.

The only reasonable way to deal with this is to avoid them all to the greatest extent possible. You have no control over whether or not you deal with Cloudflare, unfortunately, which makes them the worst of the lot.

That doesn't sound right. You shouldn't see a loading page for over a minute. If you're open to providing more details privately I'd love to help troubleshoot. You can drop me an email at amartinetti @ cloudflare.

Edit: see comment above.

Really?

> Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost.

Still, it doesn't excuse Cloudflare that there's no redress if you are caught on a block or even a clue on what you can do to reduce it (especially that Spamhaus do have redress procedures).

Basically DDOS booters use Cloudflare to protect their websites from competitors, since Cloudflare is one of the best. The same people Cloudflare is protecting (and claims to do so on an ethical neutrality basis) is furthering the need for Cloudflare to exist.

How about _also_ pointing to a knowledge base article for how an end user could go about working out what network activity from their IP might be flagging Cloudflare’s systems?

One source of that would be a blog post on your company's website that was actually authored by you! Point 2 below:

>"Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost. [1]

So it's not such a far-fetched notion is it?

[1] https://blog.cloudflare.com/cleaning-up-bad-bots/

https://developers.cloudflare.com/firewall/recipes/block-ip-...

I was surprised to learn Cloudflare was born out of Project Honeypot, so I am guessing Cloudflare does share data with them:

https://www.projecthoneypot.org/cloudflare_beta.html

This tactic is quite common among BigTech and something I've experienced with both Google and Amazon - once you are hooked onto their product, one day they will suddenly deny some aspect of their service to you and force you to share more personal data with them to get access to it. For example, Amazon will one day start to ask you to click a link sent to your mobile to access their account, or Google or Microsoft will block your account and ask you for your mobile number to "verify" you etc. When you are blocked from using a service suddenly, however privacy conscious you are, in your desperation you will be forced to comply.

I have experienced this with CloudFlare too once when many website were suddenly blocked for me by CloudFlare on all browsers and I was forced to install their extension to access some information I needed from a website urgently. I have no doubt in my mind that even otherwise, they just deliberately and randomly blocked access to some sites and displayed their "captcha" page just for PR and "brand awareness". Now that CloudFlare has realised this is backfiring on them because of the negative emotions being associated with their brand, they have now redesigned their "captcha harassment" page to give less prominence to their branding than before.

That person should start with the assumption they haven't been misclassified and eliminate the possibility that a device on their network is compromised.

I do think Cloudflare could do better here to let the owner of an IP know why they're suffering from poor reputation.

However, it's not immediately clear to me how they could accomplish this without weakening their side of the car vs. mouse game.

[1]: https://easydns.com/blog/2020/07/20/turns-out-half-the-inter...

We did have it working mostly fine for some time back in 2021 but haven't been able to since.

There are multiple open issues reporting this on the GH repo with no real follow-up from maintainers apart from maybe a "should be fixed, open again if still an issue".

ie https://github.com/privacypass/challenge-bypass-extension/is...

Probably from scam called mail blacklists

[0] https://www.cloudflare.com/bandwidth-alliance/

If two independent sites believe you are a bot, you or something at your address just might be.