At 384 threads wouldn't the CPU be heavily bottle-necked on memory?

So if I’m understanding correctly, the FL2 rewrite reduces latency by 70%. But the CPU “upgrade” increased latency by 50% from baseline. So… it’s not any faster for customers, just higher performance/watt for Cloudflare.

And this is against the alternative of potentially using the new runtime code on the old hardware which would have reduced request serving latency by up to 70%, which Cloudflare has decided not to do.

I like these transparent blog posts and appreciate them. However, essentially as a customer we are being told 100% of the savings is being eaten up by Cloudflare and the customer gets no tangible benefit. This was an engineering and management decision.

Oddly it’s a bad look for AMD too. It gives clear insight into where hyperscaler’s priorities are at in general. This is why customers are slowly repatriating their hosting services off cloud.

Eh. It depends what your bottleneck is. If the bottleneck is now, say, CPU cache contention because you've doubled your thread count, it's entirely possible that FL1 running on the new server generation is operating in a different regime than on the previous generation. You can see some hints of that happening, since doubling thread count didn't result in a doubling of throughput.

In fact, I suspect based on the throughput doubling with FL2, we're back in the same regime as the baseline.

It would be useful to see what the latency is of FL2 on Gen12 compared to baseline (FL1 on Gen12), just to confirm.

Yes fair points. The think it’s also indicative of how important it is that code be optimized for the specific hardware it will run on. Systems need to be considered and optimized as a whole. Still an interesting post.

Dupe of later post: https://news.ycombinator.com/item?id=47495795

This thread has more comments (28+)

He had 50510 karma with only 28 comments and late post lol. Skill issue