tkadlec

I built the site and I agree. :)

Two things:

1. The data is in the process of being updated. Prices change quickly and from the trend of past changes I fully expect these prices to drop as soon as I make the change.

2. While the ITU (and other data sources) try their best to look at all sources, I have to take them at their word. I've heard several times that they seem to have overlooked plan A or plan B, so it doesn't surprise me if there are other plans lurking out there.

Long-story short, it's absolutely best to consider this as a gauge/appromixation, rather than a scientific exact number.

You probably already noticed, but you can also access site cost data from the WebPageTest itself. (Note the cost column in the metric summary for your linked test, for example: https://www.webpagetest.org/result/210707_BiDcXB_65148b09266...)

Bytes over the wire (I can clarify in the post too).

So _most_ of that weight is gonna have gzip or brotli applied. (Last I looked, around 17% of JS requests recorded by HTTP Archive are uncompressed.)

I suspect you're absolutely right! :)

Using jQuery is definitely a different style of development than using something like Angular/Vue/React. I tried to clarify in the post (under "The Big Picture") that the data shouldn't be construed as me saying "React is slower than Vue or jQuery" but rather that there are characteristics of the way we build when we use React (combo of ecosystem, documentation, technical approach, etc) that leads to a lot more work happening on the device. The question is whether that's something that we're ok with.

Hopefully that came across in the post! I certainly don't want anyone thinking this is like a benchmark or anything that is pitting jQuery execution versus React execution or similar.

Hopefully that's not the way it comes off! The entire last section of the article is my attempt to make it clear that this _doesn't_ happen because we're bad people.

> So clearly, folks who have built a heavy site are bad, unethical people, right?

> Here’s the thing. I have never in my career met a single person who set out to make a site perform poorly. Not once.

> People want to do good work. But a lot of folks are in situations where that’s very difficult.

> The business models that support much of the content on the web don’t favor better performance. Nor does the culture of many organizations who end up prioritizing the next feature over improving things like performance or accessibility or security.

> What I'm getting at is maybe AMP is Google's way to make the problem easy. Sometimes when a general problem is difficult, you're better adding restrictions to make the problem tractable.

It absolutely is! (Sorry if I made it sound like it wasn't). That's what the goal is of Feature Policy and other standards too: to add restrictions to make it easier to solve.

> Can you elaborate? How easy would it be to test AMP pages for that?

I keep meaning to make an example, but real work gets in the way. :)

Basic gist though is that AMP is, again, only a fuzzy approximation of good performance. You can still mess it up.

As for testing, the best thing to do is probably test from the origin server. That way you eliminate the Google cache layer and just focus on what the format itself actually accomplishes.

There was a post recently where a guy did exactly that, and the results were far from instant: https://ferdychristant.com/amp-the-missing-controversy-3b424...

The conclusion, for anyone who doesn't want to click through, is that the vast majority of the performance benefits don't come from AMP itself. They come from A) the Google cache layer and B) the fact that Google preloads the AMP page in the background before you click on it.

First off, you're right: any metric can be gamed. Even so, Google is already using performance to factor into its algorithm, so some level of reasonable accuracy must have already been agreed upon.

Ensuring a certain level of performance is not a very easy problem to solve. But it's doable.

Myself and a few others sat down with the AMP team awhile back and chatted about what that would look like and there were definitely a few different layers that would be needed to get it to work.

The first is already in progress. There's a spec for something called "Feature Policy" (https://wicg.github.io/feature-policy/). Feature Policy would let you tell the browser you don't need certain features which would in turn let browsers take shortcuts (so to speak).

For example, you could declare that you will not be allowing document.write. Enough features being declared could serve as a way for Google (and others) to say "Ok, this is going to be reasonably fast."

There's more needed of course: we had discussions about a standard related to declaring a limit on asset sizes, etc. But it's a start. And while it will always be a little fuzzy, the same is true of AMP. It's completely possible to build a slow AMP page.

The best Google can do is say "if they're doing X, then the odds are good that it's performant".

Not by itself, but it does provide a portion of what is needed. Specifically:

> The developer may want to use the policy to assert a promise to a client or an embedder about the use—or lack of thereof—of certain features and APIs. For example, to enable certain types of "fast path" optimizations in the browser, or to assert a promise about conformance with some requirements set by other embedders - e.g. various social networks, search engines, and so on.

You're right that there's more to it though.

FWIW, they did let performance factor, slightly, into ranking, but there was never a "fast" icon (one was rumored, but never shipped). An icon like that would've easily motivated folks, no need for AMP to be involved.

That being said, there are technical reasons why this hasn't happened yet. The good news is that some web standards (ex: https://wicg.github.io/feature-policy/) are being worked on that would (hopefully) allow Google to verify performance of sites without relying on AMP.

This would give Google no excuse to give AMP content special treatment, and would hopefully relegate AMP to what it should've been since day one: a framework for performance, but one that wasn't required or bolstered by any Google-colored carrots.

Automated tooling is a must, yes. The riskiest part about relying on ONLY GH's solution (IMO) is the NVD/CVE limitation.

I agree, CVE would be _awesome_ in theory. In reality, very few file for CVE's and so the coverage is iffy (~11% of npm package vulns and about ~67% of rubygem vulns https://snyk.io/stateofossecurity/).

But it goes beyond that. There was a great paper earlier this year (https://arxiv.org/abs/1705.05347) that highlighted many other issues: lag between CVE and NVD (which is where all the useful info comes from), mismatched CPE's, nonexistent CPE's, etc.

I would love to see us get to a point where the CVE/NVD was enough, but we're far from it right now.

Completely agree! The post actually alludes to that a bit towards the end.

> Single Page Apps increase the amount of client side logic and user input processing. This makes them more likely to be vulnerable to DOM-based XSS, which, as previously mentioned, is very difficult for website owners to detect.

The more significant work we do on the client, the more interesting it becomes as an attack vector.

Ha! I'll try to add some decimal points for sub $.01 sites.

This is a very fair point, sir. I'll pull the phrase.

Yeah, it's a rounding thing. I'm guessing your site is so lightweight that the cost ends up being negligible. You should be able to click through from the result page to a fully detailed report from webpagetest.org.

> * The complete list of the 72 libraries that were tested? I could not find it.

Me either. They list out the 30 most popular of those 72, but I can't see the full list. Yet another reason why the 37% they report may be underselling the issue—without being able to see the full list, it's hard to confirm 100%.

> To summarize (please correct me if I am wrong): So, of the top 75k Alexa website, 37% use a version of one of the 72 tested javascript libraries with a known vulnerability?

Yes, that's the claim they're making.

> Are there different levels of classification in the vulnerabilities? As in, do some allow for successful MITM, do some allow for injected code, or are they more benign?

They don't go into that. Based on what I know about the vulns in the libraries they discuss, they didn't do anything to distinguish low/medium/high or vulnerability type. From what we see in our DB, XSS remains the most common type.

> This is very interesting, but I think we need a lot more data

I'm digging through our (https://snyk.io) analytics and a few other sources to try to get a different (albeit, npm-centric) perspective on this. I'll try to remember to come back and ping you when it's done.

Really depends. How big is the once CSS file you're loading and how much of the CSS is common from page to page? If 20kb is for a specific component on one or two, less-visited pages, then it probably doesn't make sense to lump everything together.

A fairly safe default starting point would be:

1. Tiny subset of inlined critical CSS (See https://www.smashingmagazine.com/2015/08/understanding-criti...) 2. Asynchronous + rel=preload loading of the main CSS of your site 3. Then, if there are hefty chunks of CSS for specific pages only, bring those in on those pages.

This changes a bit if you're using HTTP/2. Then the multiplexing makes it beneficial to break the CSS into a few smaller files instead of one big one.

In the end, you probably just have to experiment and test. :) No guaranteed "this is best" approach here.