Developers make these kinds of improvements all the time. Are you saying that it would have been impossible without AI?

That codebase existed for 20 years and had contributions from nearly 200 people.

Sure, they could have come up with those optimizations without AI... but they didn't. What's your theory for why that is?

Maybe because it’s a non issue. I saw that those improvements are in the order of micro seconds, while the transfer time of a page is measure in 1/10 seconds or even several seconds. Even a game engine have something like 15 ms to have a frame ready (60hz).

Lots of small improvements add up - the total performance improvement is 53%. That's significant.

If you're the size of Shopify that represents a huge saving in server costs and improved customer-facing latency.

> the total performance improvement is 53%. That's significant.

This percentage is meaningless on its own. It’s 4 ms shaved off a 7 ms process. You would need to time a whole flow (and I believe databases would add a lot to it, especially with network latency) and figure out how significant the performance improvement is actually. And that without considering if the code changes is not conflicting with some architectural change that is being planned.

I'll take a 53% performance boost in my template language any day of the week.

It's written by AI, so those numbers are probably fake. What numbers do you get on the benchmark?

I see no reason to run my own benchmark here, the numbers aren't run through an LLM they're right there in the JSONL file: https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d...