News for hackers. Stuff that matters.
Re:Cloudflare API Down by ryandvm on Thursday November 02, 2023 @09:33AM (#38113295)
I dunno. Cloudflare gives me the creeps. I have no idea why so many folks think large swaths of the Internet should be reliant on a single company.
Re:Cloudflare API Down by ebbp on Thursday November 02, 2023 @09:40AM (#38113375)
In principle I agree with this, but do feel this is said more readily about Cloudflare than other companies it could said about - such as Amazon (via AWS), Google and Microsoft.

Perhaps my own mental model is wrong, but I see them as a credible challenger to those very oligopolistic companies, and wished there were more Cloudflares.

Re:Cloudflare API Down by acejam on Thursday November 02, 2023 @10:57AM (#38114359)
To each their own, but I think this is said more frequently about Cloudflare because they are often playing the middleman, via their CDN service. In comparison, AWS and others are the actual origin.
Re:Cloudflare API Down by msmith on Friday November 03, 2023 @12:28AM (#38124531)
I feel the same way. What about Akamai, Fastly, or Okta? Maybe Cloudflare gets more attention because of their low end plans are accessible to anyone.
Re:Cloudflare API Down by risyachka on Friday November 03, 2023 @08:53AM (#38127966)
Its not just low end plans. Their pricing is basically the only one that feels fair. They don't charge you for bandwidth, unlike others that try to make on it as much as possible, while at the same time having other services also priced significantly higher.
Re:Cloudflare API Down by rvnx on Friday November 03, 2023 @09:59AM (#38128868)
+ (Global) Cloudflare Workers are amazing compared to Google Cloud Functions or other services that are regional, expensive and slow to start.
Re:Cloudflare API Down by viblo on Friday November 03, 2023 @11:14AM (#38130017)
They charge for bandwidth if you use enough of it on the enterprise tier.
Re:Cloudflare API Down by johnklos on Friday November 03, 2023 @09:06AM (#38128118)
The difference is that nobody complains and most people agree when you talk smack about Amazon, Google and Microsoft. The general consensus is that they're big, dumb and knowingly evil, and most of the time their actions can be explained by that.

When we talk smack about Cloudflare, such as about their hosting of phishing, their underhanded DoH stuff, their complete lack of abuse handling, et cetera, lots of people come to their defense and make excuses for them.

You can like a company's product and still think the company is big and desires to be evil, but there's an emotional component for some that makes "us versus them" knee-jerk reactions more compelling than, "hmmm... is this correct?" evaluations.

I don't think any of these Cloudflare apologists would try to argue on facts that Cloudflare isn't trying to be a monopoly, isn't trying to recentralize the Internet, isn't marginalizing the rest of the non-western world, isn't trying to establish dependencies that people and companies can't easily escape, but if they did, that'd make for some interesting discussion.

Re:Cloudflare API Down by remram on Thursday November 02, 2023 @09:45AM (#38113426)
No free (or even cheap) alternatives exist. If you have a little site that might be a DoS target, you have to use it.
Re:Cloudflare API Down by capableweb on Thursday November 02, 2023 @09:51AM (#38113497)
How many little sites do you run that get hit by DDoS? I personally run about 10 tiny websites myself, some of them have around ~1-2K daily active users, but neither of them have suffered from any DDoS frequently nor do they use CloudFlare at all. One has been hit once by a DDoS that kept trying for ~2 days to bring the site down, but a simple "ban IPs based on hitting rate limits" did the trick to avoid going down, so wasn't a very sophisticated attack.

It seems to be a common misconception that people defaults to, that you have to use CloudFlare or some alternative, otherwise you'll get hacked/ddos'd for sure.

Re:Cloudflare API Down by hombre_fatal on Thursday November 02, 2023 @10:23AM (#38113907)
You could do this with anything. X hasn't happened to me, so I bet it doesn't happen to other people, so people who take measures against X are misinformed/cargo-culting (unlike me who is conveniently the smart one in my narrative).

Most services I've built that achieved any sort of traction have dealt with some sort of DoS including large fees when I've used CDNs like Cloudfront that are susceptible to a wget loop. I default to Cloudflare because it's the only one that actually covers my ass.

Cloudflare is so successful because the internet was built naively as if abusers would never exist. Just consider how IP address spoofing is still possible today and you'll begin to realize how broken the internet has always been long before you even get into dirt cheap residential smart toaster botnets.

Re:Cloudflare API Down by capableweb on Thursday November 02, 2023 @10:26AM (#38113949)
Please do take note that I'm not saying anything like what you claim I said. I'm asking if it's something people commonly get hit by, as I myself haven't had severe issues with it.

I'm not saying others are misinformed or cargo-culting anything, just that I'm seeing lots of people who probably never get hit by DDoS in the first place (couple of visitors per day) adding CloudFlare by default as that's what they see everyone else is doing.

Of course if you do frequently get hit by DDoS attacks, there is nothing wrong with trying to protect yourself against it...

Re:Cloudflare API Down by hombre_fatal on Thursday November 02, 2023 @10:44AM (#38114176)
FWIW Cloudflare offers lots of useful services beyond DDoS protection—that's just one of them. Once you use Cloudflare for one service, it's nice to have all of your domains going through their DNS at the very least even if you were to bypass their stack.

Aside from ideological preferences or a preference for some other service, I don't see what you gain by avoiding them.

Re:Cloudflare API Down by grayhatter on Thursday November 02, 2023 @11:20AM (#38114750)
> Just consider how IP address spoofing is still possible today and you'll begin to realize how broken the internet has always been long before you even get into dirt cheap residential smart toaster botnets.

I know I'm going to regret asking but, ok, I'll bite... why does IP address spoofing prove the Internet is broken? Especially considering that a) the point of internet routing is to route packets whenever possible especially around damage. and b) by volume, the internet is TCP and you can't complete a handshake with a spoofed ip.

Re:Cloudflare API Down by sophacles on Thursday November 02, 2023 @11:30AM (#38114917)
With spoofed src addr:

* you can do all sorts of udp amplification attacks (e.g. dns - send a zone transfer request in a single packet with a spoofed IP, and the IP you spoofed to gets a lot of traffic in response.)

* you can do tcp syn or ack floods with a spoofed IP, these eat resources on the target machine. syn floods cause the os to allocate a new connection and timers waiting for the third ack.

* you can send lots of bad packets from a spoofed ip that causes automated systems to lock out those IPs as a response to attack traffic. If those lockouts block IPs that should be allowed (a type of denial of service)

And plenty more.

Re:Cloudflare API Down by grayhatter on Saturday November 04, 2023 @01:42AM (#38138501)
I'm not terribly impressed... this reads like a response from an LLM. Yeah I know what kind of packets you can send with a spoofed source IP... But the question was, given these are all decades old, how does that prove the Internet is broken?
Re:Cloudflare API Down by sophacles on Saturday November 04, 2023 @12:05PM (#38142295)
The point of the internet is to provide a robust communictions platform. If fundamental infrastructure of that communcations platform can be abused to deny communications, and further that the abuse can continue with the root cause unaddressed for decades, then the platform is broken.

The fact that routing is designed to go around damage is orthoganal to this, and has not bearing on the fact that the communications platform can be used against itself to prevent communications (via spoofed IPs).

For literal decades partial solutions to the spoofing problem have been known - rp filtering would eliminate a lot of problems yet still isn't close to universal.

BGP has been vulnerable to all sorts of simple human mistakes for decades and decade old solutions like IRR are only slowly being adopted because many of the people that run the internet are too busy pretending they are important and good at building systems to actually make the systems good. When those same simple mistakes are intentional, all sorts of IP traffic can be spoofed including full TCP connections.

The fact that there isn't a widely supported way for the consequences of spoofing to be mitigated without paying out the nose for a 3rd party service is pretty broken too. Allowing destinations be overwhelmed without any sort of backpressure or load shedding is a fundamental flaw in "get packets to destination no matter what". An AS should be able to say "I no longer want packets from this subnet", and have it honored along the entire path. This should be a core feature, not an add-on from some providers.

The internet does work as designed, however it's folly to think that the first attempt at building something so different to anything that came before it is the best way to do it and reusing to address design decisions is fundamentally broken.

Re:Cloudflare API Down by grayhatter on Sunday November 05, 2023 @12:39PM (#38153403)
> The fact that routing is designed to go around damage is orthoganal to this, and has not bearing on the fact that the communications platform can be used against itself to prevent communications (via spoofed IPs).

> For literal decades partial solutions to the spoofing problem have been known - rp filtering would eliminate a lot of problems yet still isn't close to universal.

It's orthogonal, and yet of the places where it would actually matter or have the strongest effect, it's not used? I wonder why... 'cept not really. The Internet seems to still be functioning pretty well for something fundamental broken. For the vast majority of internet routers it's entirely reasonable for them to accept any source IP from any peer. because it is impossible to prove that peer can't reach somebody else. The exception is a huge number of endpoint ISPs who shouldn't be sending these packets and it's on them to filter them. I would love a way to identify and punish these actors for their malfeasance, but I'm not willing to add a bunch of complexity to do so.

> because many of the people that run the internet are too busy pretending they are important and good at building systems to actually make the systems good.

wow, that's a super toxic comment... and I'm an asshole saying that.

> Allowing destinations be overwhelmed without any sort of backpressure or load shedding is a fundamental flaw in "get packets to destination no matter what".

one mans fundemntal flaw is another's design trade off... every single system that has ever seen widespread adoption has defaulted to open and permissive. every. single. one. it's only after seeing widespread adoption does anything ever add in restrictions and rules and most often when it does it's seen as the enshittification of something. (most often then because exerting control allows you to vampiriclly extract more value). but dropping packets one system is overloaded is exactly what the internet does do, what you're describing sounds more like TCP working around it. (poorly admittedly)

> An AS should be able to say "I no longer want packets from this subnet", and have it honored along the entire path. This should be a core feature, not an add-on from some providers.

I could not agree more. but this is a missing feature not a fundamental flaw. the internet still works for the vast, vast majority of users and as I've said in a different thread the use or dependency on cloudflare is often a skill issue not a requirement.

You're 100% correct, core internet routing has many fixable defects. and many ISPs are moving slower than could be reasonably considered ethical or competent. But for something this core of infrastructure I would actually prefer slow and careful over the break everything on a whim because of the "move fast" mind virus that has overtaken CS.

Re:Cloudflare API Down by ttymck on Friday November 03, 2023 @12:50AM (#38124662)
> IP address spoofing is still possible today and you'll begin to realize how broken the internet has always been

not that you need to have the answer to make your point, but now I am curious: what is the alternative architecture that prevents IP address spoofing? Wouldn't proving you are the IP you purport to be require some sort of authentication, which requires some centralized authority to implement? Which would require a fundamentally centralized internet?

Re:Cloudflare API Down by greyface- on Friday November 03, 2023 @01:55AM (#38125046)
> Which would require a fundamentally centralized internet?

Yes, that fundamentally central authority overseeing the IP address space exists today as IANA, which delegates to RIRs such as ARIN and RIPE, who allow ISPs to assert authority over address space cryptographically (RPKI) and/or in a central registry (IRR). This is the basis on which BGP announcements are typically filtered.

> what is the alternative architecture that prevents IP address spoofing?

It is possible to extend the same filtering approaches used with BGP to actual traffic forwarding without making fundamental architectural changes. See BCP38 (access) and BCP84 (peering). Widespread adoption of these would eliminate IP spoofing.

Re:Cloudflare API Down by cryptonector on Friday November 03, 2023 @01:09PM (#38131737)
There's two parts to the answer to GP's question. One is egress filtering, which is widely deployed, and the other is BGP security, which as you know is being deployed.
Re:Cloudflare API Down by drcongo on Thursday November 02, 2023 @10:27AM (#38113963)
Depends what you're counting as "little", but maybe your experience of 10 tiny sites has blinded you to the fact that sites for activist organisations, whistleblowing, investigative journalism, non-profits and so on, are very regularly targeted.
Re:Cloudflare API Down by sophacles on Thursday November 02, 2023 @11:00AM (#38114420)
I've never had any of my 3 houses burn down. I don't know why everyone says buy insurance, and having a fire extinguisher is a good idea.

It seems to me like a common misconception that you "have to have a fire extinguisher" (or other supression system), otherwise you'll have your house burn down.

Re:Cloudflare API Down by grayhatter on Thursday November 02, 2023 @11:27AM (#38114859)
good luck with all those strawman replies...

It only takes one trivial event to make you a cloudflare zealot, most ISPs (web hosts) don't provide attack mitigation nor prevention. If you didn't know that you could ban IPs programtically you'd have been screwed* and would have loved how quickly cloudflare would protect your site.

It's not a difficulty thing, it's an ignorance thing. cloudflare isn't better,or magic, it's easy and popular. Just like social media, and aws.

For some people, i.e. most 'web developers' you will get hacked if you don't use it... because the alternative would be literally nothing. Not everyone wants to do things correctly, they only want it to work, working correctly is required.

Re:Cloudflare API Down by duckmysick on Thursday November 02, 2023 @12:52PM (#38116435)
What's the correct way to do things if I don't want to use Cloudflare for attack mitigation?
Re:Cloudflare API Down by grayhatter on Saturday November 04, 2023 @01:37AM (#38138482)
I'm not an expert in network security, so I can only give you an idea. The correct way to do things depends on so many factors, but the TLDR n probably use multiple ways to identify malicious traffic, and then block it as early as possible. Dropping all traffic from abusive IP addresses with netfilter is where you start because it's easy/simple. Then you can move on to grouping by ASN, and dropping closer to a load balancer.

The problem (as far as I know) that cloudflare actually provides, is being able to identify abusive networks, and making them prove they're not bots before they reach your server. If you can't even identify the abusive traffic, you don't have many options other than cloudflare.

Re:Cloudflare API Down by solatic on Thursday November 02, 2023 @11:44AM (#38115135)
> How many little sites do you run that get hit by DDoS?

Do you live in a country that has enemies? DDoS attacks are one of the primary weapons of cyberwarfare.

You may not have suffered an attack yet, but thinking the world is a peaceful place in which small players have no worries is naïve.

Re:Cloudflare API Down by capableweb on Thursday November 02, 2023 @01:05PM (#38116661)
> Do you live in a country that has enemies?

Who doesn't?

> You may not have suffered an attack yet

But I literally shared in my previous comment that I have?

> thinking the world is a peaceful place in which small players have no worries is naïve

I agree, and I guess I'm lucky for not holding such opinion then.

Re:Cloudflare API Down by remram on Thursday November 02, 2023 @01:12PM (#38116780)
I personally don't usually use it, but if I ran a Mastodon instance or blog or anything with real traffic and limited bandwidth and the slightest potential to piss someone off, I would. I am definitely not surprised by the amount of people using it.
Re:Cloudflare API Down by anonzzzies on Friday November 03, 2023 @09:35AM (#38128519)
I wrote this before here, but my site (small b2b saas with a few 100 avid users from small-medium sized companies) gets hit by massive DDOSs a few times a year. The only way I can protect against that is CF bot fight. Everything else will just immediately kill the service until it's over. The last one lasted 24 hours; there were millions of requests from 100000s unique ips over that time; many ips from azure, gcp and aws. Why? I don't have a clue but with CF you simply notice nothing at all.

I cannot rate limit on the machines itself as they die immediately, so then I need to get more advanced firewalls etc which are vastly more expensive than CF.

Re:Cloudflare API Down by r1ch on Thursday November 02, 2023 @01:23PM (#38116968)
OVH's DDoS protection works great and is included by default on all servers. It's blocked hundreds of attacks for us and the time to mitigate is only a few seconds.
Re:Cloudflare API Down by jojobas on Friday November 03, 2023 @09:38AM (#38128562)
Joshua Moon is laughing at you.
Re:Cloudflare API Down by jjav on Thursday November 02, 2023 @12:51PM (#38116403)
> I have no idea why so many folks think large swaths of the Internet should be reliant on a single company.

It's not just the reliance, but the fact that cloudflare is a MITM attacker (by design) on vast amounts of TLS traffic. TLS if used properly gives you end to end security, but if you use cloudflare they have access to all your cleartext traffic.

Re:Cloudflare API Down by NicoJuicy on Thursday November 02, 2023 @02:16PM (#38117798)
Which can be said from any cloud provider/hoster.
Re:Cloudflare API Down by michaelmior on Friday November 03, 2023 @08:59AM (#38128036)
How so? If I'm hosting a server somewhere and clients directly connect to my server to establish a TLS connection, failing any vulnerabilities in the implementation, there's no MITM happening and the provider can't see the plaintext traffic. (Of course, since the server needs the certificate, the provider could in theory extract that certificate and establish a MITM proxy, but this isn't by design.)
Re:Cloudflare API Down by jodrellblank on Friday November 03, 2023 @09:06AM (#38128127)
Any VPS or virtual server cloud provider can potentially see the plaintext traffic - it's in plain text of the memory of their hardware and they could be looking at it. They technically could be scraping your SSL keys from memory, or scraping your SSL private key from disk (if unencrypted storage) and then decrypting a mirror of the network traffic elsewhere. That wouldn't be MITM but you are only protected from it if you are hosting your own physical server somewhere.

"End to end security" mentioned above is limited security when "your" endpoint is owned by and controlled by someone else.

Re:Cloudflare API Down by quinncom on Friday November 03, 2023 @05:06PM (#38135016)
Here’s an example of MITM by interception of automated certificate renewal downstream of a VPS hosted at Hetzner. The presumption is that it was a lawful intercept installed within Hetzner or one of their internet providers. https://news.ycombinator.com/item?id=37955264
Re:Cloudflare API Down by michaelmior on Friday November 03, 2023 @08:53AM (#38127968)
It's not an "attacker" by design, but certainly MITM.
Re:Cloudflare API Down by NicoJuicy on Thursday November 02, 2023 @02:14PM (#38117781)
Tbh. There are multiple objective reasons for me why they don't.

2 outages in the last week is 1 objective reason why they do, though.

The first one did get fixed in 30 minutes, which is probably some sort of record. I can't remember where any other cloud provider updated their status page even within 30 minutes ( or they hid it within their authenticated environment)

Re:Cloudflare API Down by is_true on Thursday November 02, 2023 @02:51PM (#38118266)
I feel they have one of the most fair offerings, not trying to squeeze every last penny from you like other cloud providers. Just my opinion of them, probably wrong.
Re:Cloudflare API Down by NicoJuicy on Thursday November 02, 2023 @03:34PM (#38118821)
I don't think you're wrong.

Reference: CEO takes less than average salary... 600 k./yr. After > 14 years.

Note: as far as I know ( which is one of the many reasons I do "believe" in cloudflare), he is already wealthy and didn't start cloudflare for the money.

The facts seem to be supportive of that statement ( cfr. Compensation).

The previous time I was a fan of a company was AMD in 2014 ( somewhere around that time) and because of Lisa Su + product line up.

Re:Cloudflare API Down by michaelmior on Friday November 03, 2023 @08:55AM (#38127991)
> I have no idea why so many folks think large swaths of the Internet should be reliant on a single company.

I don't think it's really that so many people think Cloudflare should be relied on. It's that Cloudflare generally has a good track record and their basic services are available for free. Actually, I don't know of any service similar to Cloudflare with such a generous free tier.

Re:Cloudflare API Down by ActionHank on Friday November 03, 2023 @10:04AM (#38128935)
I think you're underestimating just how many devops and infra people dip into the mainstream tech news and adopt that as their new religion.

I have had multiple conversations with high level individuals about why we should be using CloudFlare so widely and what the fallback is if there is an outtage. Usually it boils down to "because reasons".

Today it's #notmyproblem and life is great.

Re:Cloudflare API Down by jodrellblank on Friday November 03, 2023 @09:08AM (#38128161)
> "I have no idea why so many folks think large swaths of the Internet should be reliant on a single company."

Who thinks that? Can you link to anyone who has said that?

Downvoted for "I am superior to <strawman>" comment.

Re:Cloudflare API Down by richbell on Friday November 03, 2023 @09:16AM (#38128256)
> Downvoted for "I am superior to <strawman>" comment.

I didn't interpret their comment this way. To me, it read "this thing gives me bad vibes and I don't understand why so many people like it."

Re:Cloudflare API Down by jodrellblank on Friday November 03, 2023 @11:20AM (#38130139)
"I don't understand why people like it" is very different from "so many folks think large swaths of the Internet should be reliant on a single company". Take Chrome browser; it's fine to use FireFox because you think it's better, it's fine to use FireFox even though you think it isn't as good but you'll take the mild inconvenience on the principle that the internet shouldn't be dependent on a single company. It's also fine to use Chrome because you think it's better, but nobody - absolutely nobody, anywhere, ever[1] - who chooses Chrome does so because they think the internet should be reliant on a single company for a web browser and they want to support making that happen.

"I don't use Chrome, I don't know why everyone thinks the internet should depend on Google" is a strawman because nobody does think that; many people use Chrome despite thinking the exact opposite of that, even. Same with CloudFlare, it's free, it's convenient, it's very good at what it does (current outage excepted), it's widely known, easy to work with, has good support. Nobody who chooses it does so because they want to hand internet control to a single company. And "I don't know why people use (popular, well known, well made solution)" is a very common internet comment which communicates a certain message.

[1] people who work for Google are paid to think that, so their decision doesn't count.

Re:Cloudflare API Down by richbell on Friday November 03, 2023 @11:57AM (#38130660)
You're right in the literal sense, but I don't think they meant that literally (maybe they did, who knows).

I personally haven't seen anyone praise the Internet being reliant on a single company. However, I have seen lots of praise for Cloudflare over the years and "naysayers" (such as people raising concerns about them MITMing half the Internet) being aggressively downvoted. In that context, I see it as many people tacitly endorsing Cloudflare and not caring about the control it holds, rather than people explicitly saying "Cloudflare _should_ control the majority of the Internet."

Re:Cloudflare API Down by cryptonector on Friday November 03, 2023 @01:05PM (#38131682)
What about them gives you the creeps?
Re:Cloudflare API Down by oliwarner on Friday November 03, 2023 @07:04PM (#38136220)
No idea?

They came along, out of nowhere and started offering their infrastructure, their global distribution network for free as a reverse proxy. This allowed people to scale their single-server-services out for nothing, it insulated servers from DDoS and single-government action. For the people that needed it, it took 10 minutes and revolutionised a lot of slow websites.

But then they started offering actual services. A formal CDN, largely for free but after that, pennies on the dollar of what major players were asking. And 6 years ago they started building your stuff for you, allowing you to host it near your users. They sell domains at cost, host DNS for nothing, and handle inbound email for you.

As a webdev, they're making my life very simple. Things that took me a day to bash out and bootstrap for a new client, I've done with CloudFlare while the client is on the phone.

If they vanished tomorrow, it'd be a wrench. But that's true of so much online infrastructure. Where would you be without Github, NPM, PyPI, dockerhub, etc? Enjoy it while we can.