"HTTP/1.1 is a delightfully simple protocol, if you ignore most of it".

    A Vary field containing a list of field names has two purposes:

    1. To inform cache recipients that they MUST NOT use this response to satisfy a later request unless the later request has the same values for the listed header fields as the original request (Section 4.1 of [CACHING]) or reuse of the response has been validated by the origin server. In other words, Vary expands the cache key required to match a new request to the stored cache entry.

    [...]

    When a cache receives a request that can be satisfied by a stored response and that stored response contains a Vary header field (Section 12.5.5 of [HTTP]), the cache MUST NOT use that stored response without revalidation unless all the presented request header fields nominated by that Vary field value match those fields in the original request (i.e., the request that caused the cached response to be stored).

So, Cloudfare doesn't actually implement HTTP/1.1: you can't just decide to ignore parts of the standard and still claim you implement it, that's only allowed for "SHOULD [NOT]" and "MAY" parts, not for the "MUST [NOT]" ones.

Here's the RFC ref for anyone wanting to check it out: https://www.rfc-editor.org/rfc/rfc9110#field.vary

Notice that this is the HTTP "Semantics" RFC, not the "core" one (Message and Syntax) which is the venerated RFC-7230 [1] which is indeed quite simple for such a widely used protocol.

This RFC only defines a handful of "header fields", almost all of which necessary for actually being able to "frame" (find the beginning and end, applying decoding if specified) the HTTP message : https://www.rfc-editor.org/rfc/rfc7230#section-8.1

[1] https://www.rfc-editor.org/rfc/rfc7230

See, the caching mechanism in HTTP/1.1 is actually quite nicely designed in the sense that you can completely ignore it: don't look at any caching-related fields, always pass the requests up and the responses down — and it will be correct behaviour. But when you start to implement caching, well, you have to implement it, even the inconvenient but mandatory parts.

Yes, I know (and personally experienced) that most clients forget to include "Vary" field, or don't know that it exists, or the problem that this header solves exists) but when they do include it, they actually mean it and rely on it being honoured, one way or another.

PS. By the way, 7230 is obsoleted, you're supposed to use 9112 now.

RFC-9112 is the current one, you're correct. But RFC-7230 is the "venerable" one and always will be.

Surely you mean RFC-2616? That was around for 15 years, including all of the 2000s which were the most formative Web 2.0 years; the RFC-7230 came by around 2014 and lasted only 8 years OTOH.

Ok, both are :)

There is also nothing in the HTTP spec that allows a cache to randomly return a CAPTCHA instead of the cached content.