There are also new financing methods, paywall implementations, prices, cooperations, etc. being tested all the time, with some successes and many failures.
While people here and in general often seem to consider publishers to be rather stupid, I think there's enough evidence at this point to conclude that it has become a particularly difficult business, what with almost half of revenue almost instantly disappearing when classifieds move to the net, and people generally considering to appear out of thin air because the costs of copying are zero.
They are running randomized experiments, but they are not necessarily running the right experiments. If the bottom line is revenue, then attrition may not be directly measured; if they are testing on per page-view or per session instead of bucketing users permanently, then they can miss effects too; and if they are testing by bucketing users simultaneously, they can miss non-additive site-wide effects. If you look at my comment, I link two publicly-documented randomized experiments, and both set up/analyze their tests in an unusual way.
After I published my results, I heard about some parallel results, but not that many. I heard whispers that Google might (or might not) have internally run equivalent tests; I heard from Chris Stucchio that an unspecified company he worked for had run an A/B test with similar results; I heard from a mobile developer that they had run a similar experiment but found no effect of ads; Pandora subsequently published their A/B test; and Mozilla has published this. I also read through most of the 'advertising avoidance' literature in academia going back 30 years, and found nothing relevant. And that's it.
So, if everyone is always continuously running obviously relevant experiments... where are they?
Also most of the innovation happens with adtech vendors who try out new ways to get attention and payments. TrueX and Sourcepoint have been working on some new stuff, along with the various formats that keep being invented.
Publishers like Google, Microsoft, and Yahoo do publish many research papers going in depth on A/B tests - such as the effects of page load time - and I've benefited from reading them over the years. So yes, it is something they do.
For example, Washington Post runs experiments specifically with ads as a digital media/news publisher, but you won't see that research published anywhere.
Google, Microsoft (Bing), and Yahoo are not considered media publishers because they are ad networks first.
If you just dismiss actual media publishers like NYT and BBC as small fry (when they are some of the biggest) then I'm not sure what you were expecting as an answer.
Even if you don't discount their sizes, they do not reveal their research because there's no time or incentive to reveal things like yield management, which is complex, audience specific, and contains competitive intel. That's why it can't be officially found, but you can get details if you talk to their dev teams.
Which is a strange thing to claim since what you claim never happens, happens all the time with some of the largest media publishers in the world. Think of how many pages Google serves via AMP, or Yahoo's front page, or Google News, quite aside from their companies literally being based on advertising. If they are not media publishers who are doing experiments of the relevant sort, no one is and you are blatantly no-true-Scotsmanning.
There are large entities, better positioned than anyone else in the world to know, who run experiments constantly, and publish the results constantly.
So.
Where are they?
> they do not reveal their research because there's no time or incentive to reveal things like yield management, which is complex, audience specific, and contains competitive intel.
Congratulations, you have disproved the existence of hundreds or thousands of papers and blog posts. I must be hallucinating them all.
> That's why it can't be officially found, but you can get details if you talk to their dev teams.
Really? Then why, when I talk to them, I never hear any specifics, I always get told either they don't know or no one's checked, and only once in a blue moon do I hear something relevant (always without details)? Since you apparently talk to so many dev teams and they tell you all about the effects of ads on user retention and total traffic from the many experiments they run but never ever publish on, perhaps you could summarize them and provide the details for the rest of us.
No, Google, Microsoft, Yahoo are not considered media publishers by any actual media publisher; because they are ad networks that do some news aggregation at best and do not produce original content. There absolutely is a definition that you are refusing to recognize for some reason.
>> There are large entities..., who run experiments constantly, and publish the results constantly. So. Where are they?
Again, what are you referring to here? The big companies are not publishers and what they do is closer to Amazon.com testing page speed rather than NYT testing ad load. Actual media publishers do not publish results constantly, if ever.
>> Congratulations, you have disproved the existence of hundreds or thousands of papers and blog posts. I must be hallucinating them all.
What? How are you asking about the lack of research and then saying there's research everywhere? Again, what are you talking about here? What media publisher* is printing heavy insights into their ad yield management?
>> ...when I talk to them, I never hear any specifics, I always get told either they don't know or no one's checked...
Because it's internal intel. Do you know anyone on the revenue team? They might be more open to details but won't know the technical background. You can look at trade publications like Digiday, Adweek, Mediapost, Adexchanger and others to find details and some rare whitepapers.
>> perhaps you could summarize them and provide the details for the rest of us.
You need to have specific questions, but here are some details:
Top sites are focused on revenue over UX. Sites have found equilibrium between ad load and retention with maximum time spent. Users rarely stop visiting sites, they just come back with adblockers instead, which has led to strategies like free articles per month and increased direct-sold campaigns and native ads from brand studios (for pubs that can afford a team). WSJ and other pubs have stopped the google-search free entry workaround as well.
UX changes include never-ending feeds and auto-changing articles to minimize bounces, along with less ad slots but more ad refreshes to increase total impressions. Tech-forward pubs like WaPo have built their own adtech stacks to show users different formats and ad frequencies depending on how they respond. Subscribers get less ads until churn is minimized. No ads for subs just leaves money on the table and even FT.com runs plenty of campaigns. Video has also invaded everything with 5x higher prices than banners.
Overall pubs have managed to squeeze out the same number of impressions from just the non-adblocked market, while optimizing paid conversions for adblockers and capitalizing on their uninterrupted attention to show more high-touch direct campaigns. Readership is up all around and both segments make more money than before.