I've worked with brilliant conversion optimizers, in a single vertical niche where we had > 30 websites, each doing between 200K-2M visits/month. We would have loved to test + copy, but every time we ran this experiment, we found huge differences.
One subtle nuance is that "generalizable" needs to be more specific than "this change increased conversions". The true goal of optimization is not the find the B that beats A; it's to find the global optimum.
The Conversion industry tends to similar language as the complementary/alternative medicine space. "I did this thing, it had this outcome. I believe this happened because XYZ. It worked for me, it will work for you!" (People telling stories which may or may not be true.)
That is also why clinical trials for interventions is so important.
FWIW, that's the goal of global optimization, a particual type of optimization, and one that rarely anybody on the market cares about wrt. business. Local optimization, "B beats A", is more frequently used, as usually all the business cares/can afford to care about is to find a C that beats B, if their competitors found the B that beats A.
If it was such a brilliant idea, DoorDash & GrubHub should change their sites so that links from their search results only take you back to their search page.
First, a custom landing page can both provide you with the restaurant and an option to search for more restaurants.
Secondly, if the goal was an optimized experience, why wouldn't they want that on their restaurant pages, even the ones that don't come in from search? There's a reason.
> One misconception I see about conversion optimization is that it's generalizable. One misconception I see about conversion optimization is that it's generalizable.
YMMV of course, but this one stretches credibility.