News for hackers. Stuff that matters.
Re:CDNs and Centrality by EE84M3i on Saturday July 03, 2021 @08:57PM (#27726444)
This article slightly misunderstands key details from the akamai blog post[1].

>Akamai experienced an outage for one of its Prolexic DDoS services (Routed 3.0) starting at 4:20 AM UTC.

Prolexic "Routed 3.0" is not a CDN product, and is generally separate from Akamai's other services (CDN, DNS). It is a layer 3 DDOS scrubbing product that works by advertising BGP routes for customer IP addresses filtering out attack traffic, and then routes the traffic to the real destination over GRE tunnels.[2]

This explains why the customers effected were only customers who are "always on" like banks, who are wary of using a layer-7 CDNs because they don't want someone else terminating their TLS connections. Prolexic doesn't make your site faster, it makes it slower.

In AWS terms: the issue was with "Shield" not with "Cloudfront". If Akamai's actual CDN went down, the impact would have been larger than Fastly's outage.

[1]: https://blogs.akamai.com/2021/06/akamai-provides-prolexic-dd...

[2]: https://www.akamai.com/us/en/multimedia/documents/product-br...

Re:CDNs and Centrality by mike_d on Saturday July 03, 2021 @11:53PM (#27727111)
It is a fully integrated core service built on the Akamai edge platform when you are buying and implementing it, and then a tiny little no-nothing "gosh are people still using that?!" in the corner when it goes down.

You also might want to take a look at who Geoff is before claiming that he is misunderstanding how a routing product works. https://www.internethalloffame.org/inductees/geoff-huston

Re:CDNs and Centrality by EE84M3i on Sunday July 04, 2021 @12:35AM (#27727247)
I'm sure he understands how it works, but IMO calling Prolexic a CDN is wrong.
Re:CDNs and Centrality by youmustbeblind on Saturday July 03, 2021 @11:06PM (#27726939)
Does anyone adopt akamai for greenfield nowadays? Seems a dated product, it's either fastly or cloudflare or CloudFront it seems nowadays
Re:CDNs and Centrality by booi on Sunday July 04, 2021 @01:07AM (#27727365)
Akamai’s edge performance is still first-class but I do think QUIC/http 2/http 3 have significantly reduced the benefit over more centralized CDN offerings.
Re:CDNs and Centrality by calpaterson on Sunday July 04, 2021 @02:44AM (#27727723)
I don't agree that the network effect in CDNs is so huge. Instead of consolidating, I think the CDN market is slowly fracturing. 10 years ago, there was only one workable CDN for most: Akamai (founded 1998). Now there is Cloudfront (launched 2008), Cloudflare (founded 2009), Fastly (founded 2011) and a long tail of others. It takes some years for a new CDN to sign big corporate contracts but the trend seems to be toward more CDNs and a number have been founded recently.

It is easier than ever for them to get started because they can begin by piggybacking on one of the existing CDNs. Eventually they will build their own network of POPs (local caches). I also don't think you need as many POPs now to build a saleable CDN as in the past - there is some fraction of the market for which as low as 5 POPs might be ok.

Re:CDNs and Centrality by youngtaff on Sunday July 04, 2021 @03:53AM (#27727991)
There may be more CDNs but most of them aren't that great…

CloudFront has HTTP/2 prioritisation, and concatenation issues

Many of the others build on nginx which also has HTTP/2 prioritisation issues

Akamai, Cloudflare & Fastly seem to be the only CDNs that are seriously investing in their infrastructure and their platform — all use custom edge server software (yes some of it forked from open source), have built in image optimisation services, have ability to run workloads on the edge etc.

Then there's the peering arrangements that have to be negotiated (and paid for if you're not pushing significant traffic)

Re:CDNs and Centrality by rajnathani on Sunday July 04, 2021 @02:55AM (#27727776)
Conversely, the usage of CDNs has increased since the 'Akamai-only' days. Case in point: The vast population of WordPress and other web applications wouldn't generally use Akamai in the 2000s, however, today CDN usage is more commonplace.

> It is easier than ever for them to get started because they can begin by piggybacking on one of the existing CDNs. Eventually they will build their own network of POPs

This increases centrality, as the point of failure is the parent CDN.

Re:CDNs and Centrality by calpaterson on Sunday July 04, 2021 @03:13AM (#27727850)
Agreed - the usage has risen hugely but that doesn't mean that it won't continue to fracture. The barriers to entry in running a CDN are not enormous and lots of startups are able to enter.

> This increases centrality, as the point of failure is the parent CDN.

Only for a while, as they are starting up

Re:CDNs and Centrality by rajnathani on Sunday July 04, 2021 @03:25AM (#27727894)
Creating POPs is not as trivial as creating a Heroku or Render like service on top of public clouds. While there are EC2s and public cloud VMs in various geographically distributed locations around the world, the instances in the actual POP regions are usually inaccessible.

Example:

- AWS has 25 regions [0]

- AWS CloudFront has 225 POP regions [1]

Partnering with Equinix/etc. and other ISPs to establish more POPs is an expensive and time-consuming process. Unless the CDN has a good edge-close-to-cities presence and a high number of those, users will generally not see value in the CDN.

[0] https://aws.amazon.com/about-aws/global-infrastructure/

[1] https://aws.amazon.com/cloudfront/features/?nc=sn&loc=2&what...

Re:CDNs and Centrality by calpaterson on Sunday July 04, 2021 @08:58AM (#27729294)
> Creating POPs is not as trivial as creating a Heroku or Render like service on top of public clouds.

I didn't claim that it was.

> Unless the CDN has a good edge-close-to-cities presence and a high number of those, users will generally not see value in the CDN.

A home truth is that most CDN users don't give a monkeys about per-city POPs. If you just had one POP per continent that would probably be sufficient to cut latency by ~75% for cache hits. Many real world CDNs have <50 POPs (eg CDN77, which has 33) and some have <25 (eg BunnyCDN sells a package with just 8 POPs). These businesses have already existed for some years and have a real customer bases.

Re:CDNs and Centrality by rajnathani on Sunday July 04, 2021 @01:25PM (#27731198)
Sorry it was ambiguous. By “users”, we mean the business buyers of the CDN service. Eg: As a developer, I don’t know where exactly my users are, if thus I was spending the same, then I would rationally prefer a CDN which has more POPs than one which has fewer.
Re:CDNs and Centrality by calpaterson on Sunday July 04, 2021 @02:00PM (#27731443)
It wasn't ambiguous, I meant buyers.
Re:CDNs and Centrality by ralph84 on Sunday July 04, 2021 @12:42AM (#27727270)
By its very nature only a few players will be able to offer (effective) DDoS protection. They are in a constant arms race for bandwidth with botnets, and with the internet of shit botnets aren't getting smaller anytime soon.
Re:CDNs and Centrality by tyingq on Saturday July 03, 2021 @09:57PM (#27726708)
There does seems to be less diversity across the board as services like DNS, SSL certs, etc, all converge into a few big players.

Not to say there aren't choices and smaller providers. Just that a higher percentage of the business is going to the biggest players than in years past.

Re:CDNs and Centrality by pm90 on Saturday July 03, 2021 @10:02PM (#27726731)
The fact that the stock price of both akamai and fastly increased after these incidents makes it seem like these incidents don’t matter. I wonder what’s the threshold at which market sentiment flips from “everybody makes mistakes” to “these providers are incompetent and will lose customers”.
Re:CDNs and Centrality by slyall on Saturday July 03, 2021 @10:47PM (#27726877)
I think in fastly's case they were not very well known and suddenly the market became aware that they were actually a big deal with lots of big customers. Probably a one-off event.

Note: I am a fastly customer, reasonably happy.

Re:CDNs and Centrality by midev on Saturday July 03, 2021 @11:42PM (#27727070)
> makes it seem like these incidents don’t matter

They don't really matter, to be honest.

If anything, they give insight into the growth of companies, showing how many key players use them. Fastly goes down and you're like damn, all these people using Fastly... I had no idea.

Re:CDNs and Centrality by viraptor on Saturday July 03, 2021 @10:47PM (#27726875)
Stock price is not a great place to look at for these incidents, I believe. It includes people who think the incident means the company is overvalued, people playing against the first, and people who think it matters in general but not for the company survival/customers long term.

In the end, the views may balance out, or even the "I'll make a quick buck shorting / buying on bad news" sentiment may actually raise the price.

(In other words, if "everyone knows" where the price will move today, the price won't move unless something is really affected long-term)

Re:CDNs and Centrality by netr0ute on Saturday July 03, 2021 @10:25PM (#27726818)
> I wonder what’s the threshold at which market sentiment flips from “everybody makes mistakes” to “these providers are incompetent and will lose customers”

When some internal document leaks that reveals "internal struggles" or some master plan, akin to the Panama Papers. The problem is, the Panama Papers are just about forgotten today.

Re:CDNs and Centrality by dannyw on Sunday July 04, 2021 @04:15AM (#27728079)
Stock prices do not go down when interest rates are near zero; simply because investors have no choice but to buy stocks.

If the Fed fund rates were 2% or 5%, FSLY and AKAM would look very differently.

Re:CDNs and Centrality by Santosh83 on Saturday July 03, 2021 @11:58PM (#27727129)
Stock price will dip if the market thinks the company is not doing well or projected to not do well. Nothing else is in consideration.
Re:CDNs and Centrality by toomanybeersies on Sunday July 04, 2021 @05:59AM (#27728492)
For me, the threshold is when they actually start losing customers (or more specifically, losing revenue).