Assignments are also almost certainly sticky based on a browser cookie.
I’m sure you would agree that would be a good representation.
Imagine you're running an AB test on a website, where the number of customers you get increases linearly over time. If you test two features sequentially, there is no duration you could pick that is small enough to where you would not expect the second option to win.
There's a peak 30-min in the morning just before work starts for most people, for example.
1. My scraper runs every 5 minutes, with a randomly-generated user-agent and never sends cookie headers
2. The charts are bucketed by half-hour periods, so even if the headline flips back and forth many times in half an hour, the colors are grouped together
3. Agreed that in the SpaceX situation (and maybe the Cuomo situation) the headlines change because the stories change. But, e.g., in the Meghan Markle situation, the first headline appears _after_ the interview is over. But that's something to watch out for!
And charts like this one[0] look (to me) like a clear example of A/B testing. But would be interested to hear other explanations!
[0] https://nyt.tjcx.me/articles/1163e0c4-e609-5cfa-aff1-b0945f1...
They may also conduct certain A/B tests only on cookied users, in which case your cookieless bot would not have seen everything.
It would have been more interesting to see the title flipping back and forth, because that would reveal how long those tests last. Less neat though.
Edit: The only example of clickbait here is OP's title.
But the quote included suggests the NYT does do the latter: "Half of readers will see one headline, and the other half will see an alternative headline, for about half an hour."
So given that the author mostly observes long consistent blocks of time with the same headline, that suggests the NYT is allocating them to a subgroup in a persistent way (by IP or whatever). Then perhaps the cases where they didn't observe A/B testing were just cases where they were randomized into the optimal (hence final) headline subgroup by accident at the beginning, and never saw any different.