News for hackers. Stuff that matters.
Re:Cloudflare outage should not have happened by dfabulich on Wednesday November 26, 2025 @12:31PM (#46060016)
That's entirely right. Products have to transition from fast-moving exploratory products to boring infrastructure. We have different goals and expectations for an ecommerce web app vs. a database, or a database vs. the software controlling an insulin pump.

Having said that, at this point, Cloudflare's core DDOS-protection proxy should now be built more like an insulin pump than like a web app. This thing needs to never go down worldwide, much more than it needs to ship a new feature fast.

Re:Cloudflare outage should not have happened by jacquesm on Wednesday November 26, 2025 @12:32PM (#46060039)
Precisely. This is key infrastructure we're talking about not some kind of webshop.
Re:Cloudflare outage should not have happened by simlevesque on Wednesday November 26, 2025 @12:39PM (#46060112)
Yeah but the anti-DDOS feature needs to react to new methods all the time, it's not a static thing you build once and it works forever.

An insulin pump is very different. Your human body, insulin, and physics aren't changing any time soon.

Re:Cloudflare outage should not have happened by jacquesm on Wednesday November 26, 2025 @01:05PM (#46060473)
You are simplifying the control software of an insulin point to a degree that does not match reality. I'm saying that because I actually reviewed the code of one and the amount of safety consciousness on display there was off the charts compared to what you usually encounter in typical web development. You also under-estimate the dynamic nature of the environment these pumps operate in as well as the amount of contingency planning that they embody, failure modes of each and every part in the pump were taken into consideration, and there are more such parts that you are most likely aware of. This includes material defects, defects as a result from abuse, wear & tear, parts being simply out of spec and so on.

To see this as the typical firmware that ships with say a calculator or a watch is to diminish the accomplishment considerably.

Re:Cloudflare outage should not have happened by joshuamorton on Wednesday November 26, 2025 @01:16PM (#46060620)
I had a former coworker who moved from the medical device industry to similar-to-cloudflare-web software. While he had some appreciation for the validation and intense QA they did (they didn't use formal methods, just heavy QA and deep specs), it became very clear to him very clearly that those approaches don't work with speed-of-release as a concern (his development cycles were annual, not weekly or daily). And they absolutely don't work in contexts where user-abuse or reactivity are necessary. The contexts are just totally different.
Re:Cloudflare outage should not have happened by jacquesm on Thursday November 27, 2025 @03:32AM (#46067071)
It is perfectly possible to engineer for faster cycles without losing control over what your code can and can not do. It is harder, for sure. But I do not think it is a matter of this absolutely not working, that's black-and-white and it never is black and white, it is always some shade of gray.
Re:Cloudflare outage should not have happened by mikestorrent on Wednesday November 26, 2025 @06:25PM (#46063469)
Thanks for spelling that out. It's so often tempting to be reductionist about things, but there is often a tremendous amount of thankless engineering inside products that we are privileged to consider as being somewhat boring. It takes a lot of work to make something so dynamic and life-critical and make it reliable enough to be considered simple, when it's anything but.
Re:Cloudflare outage should not have happened by MichaelZuo on Wednesday November 26, 2025 @07:20PM (#46063908)
If humans beings had a small chance to transform into say quadrupeds or suddenly grow tenatacles, extra hearts, organs, etc., in any given year… then wouldn’t designing a safe insulin pump literally be impossible?
Re:Cloudflare outage should not have happened by jacquesm on Thursday November 27, 2025 @03:21AM (#46067002)
Compared to what they are already doing it would be marginally more difficult.
Re:Cloudflare outage should not have happened by syockit on Wednesday November 26, 2025 @10:43PM (#46065350)
The point still stands. The human body still isn't going change. That's why insulin pump can afford to have all kinds of rigorous engineering, while web-facing infrastructure on the other hand needs to be able to quickly adapt to changes.
Re:Cloudflare outage should not have happened by jacquesm on Thursday November 27, 2025 @03:23AM (#46067013)
> That's why insulin pump can afford to have all kinds of rigorous engineering, while web-facing infrastructure on the other hand needs to be able to quickly adapt to changes.

The only reason we have a web in the first place is because of rigorous engineering. The whole thing was meant to be decentralized, if you're going to purposefully centralize a critical feature you are not going to get away with 'oh we need to quickly adapt to changes so let's abandon rigor'.

That's just irresponsible. In that case we'd be better off without CF. And I don't see CF arguing this, in fact I'm pretty sure that CF would be more than happy to expend the extra cycles so maybe stop attempting to make them look bad?

Re:Cloudflare outage should not have happened by saghm on Wednesday November 26, 2025 @02:00PM (#46061114)
All the more reason to be careful about relying on humans to avoid making mistakes when changing it rather than moving quickly and letting things fail in production.
Re:Cloudflare outage should not have happened by wathef on Wednesday November 26, 2025 @12:51PM (#46060282)
an insulin pump is a good metaphor; insulin as a hormone has a lot of interactions and the pump itself, if not wanting to unalive its user, should (most do not) account for external variables, such as: exercise, heart rate, sickness, etc. these variables are left for the user to deal with, and in this case, is a subpar experience in managing a condition.
Re:Cloudflare outage should not have happened by b-man on Friday November 28, 2025 @02:12PM (#46081777)
Indeed. I was trying to make that point on my concluding paragraph.
Re:Cloudflare outage should not have happened by Aperocky on Wednesday November 26, 2025 @12:42PM (#46060149)
> This thing needs to never go down worldwide

Quantity introduce a quality all of its own in terms of maintenance.