Investigating how the New York Times A/B tests their headlines

Re:Investigating how the New York Times A/B tests their head... by tomjcleveland on Monday March 15, 2021 @09:57AM (#26466144)

OP here. I don't know if anyone reads old HN threads but thought I'd clarify some things:

1. My scraper runs every 5 minutes, with a randomly-generated user-agent and never sends cookie headers

2. The charts are bucketed by half-hour periods, so even if the headline flips back and forth many times in half an hour, the colors are grouped together

3. Agreed that in the SpaceX situation (and maybe the Cuomo situation) the headlines change because the stories change. But, e.g., in the Meghan Markle situation, the first headline appears _after_ the interview is over. But that's something to watch out for!

And charts like this one[0] look (to me) like a clear example of A/B testing. But would be interested to hear other explanations!

[0] https://nyt.tjcx.me/articles/1163e0c4-e609-5cfa-aff1-b0945f1...

[ Parent | on HN ]

Re:Investigating how the New York Times A/B tests their head... by rattray on Friday March 19, 2021 @12:11PM (#26515049)

Did you send from the same IP? (Worth including in the post either way).

They may also conduct certain A/B tests only on cookied users, in which case your cookieless bot would not have seen everything.

[ Parent | on HN ]

Re:Investigating how the New York Times A/B tests their head... by remram on Friday March 19, 2021 @06:11PM (#26518655)

People read week-old posts on Fridays, when the hacker newsletter comes out. Thanks for the clarification!

[ Parent | on HN ]