One time I was at this silly daylong corporate team building event where they assigned us to groups and had us do "experiments." As it happened one of my group members has a PhD in Particle Physics and the look on his face was something else. Needless to say he didn't bother telling the take charge type that her experiment design was entirely nonsensical, because why bother, but I was amused.
However, it also seems as if a tuned LLM could look at the corpus of my on-line profile (mail, FB, HN) and produce a model of my attention and commentary (positive/negative). Then that model (or several models) could be tested against various ads to determine which are more likely to produce positive responses. Right now I think that would be too expensive except for very high value ads/targets, but perhaps pricing will come down and non-keyword based advertising might appear.
LLMs are useless in this domain.
I always figured there are some verticals where the value per acquisition was high enough, and the product narrowly-scoped enough, where it might make sense to microtarget individual customers-- profiling them individually and building custom messages for each of them.
The obvious target would be the narrow-scope prescription drugs-- all those ads that say "If you have XYZ-positive, PDQ-negative Exploding Sphincter Syndrome Type XVI with Sprinkles and Cheese Sauce". Presumably there are only a few hundred people with that condition in the country, but you're spending a lot of money to reach everyone else to try to catch them in your net.
Super-premium luxury goods might also fit there, but I wonder if the brand risk would be too high. Rolls Royce probably knows the 200 customers most likely to buy their new model, but it might not fit their brand to actually put together a campaign that seemed like they were actually TRYING to sell cars.
I think there are high value customers for whom this probably makes sense, but only if those doing the targeting make money from it. Right now you're the product, but the supplier is the one doing the targeting (or not) and the customer doesn't know enough about their potential customers to target. The suppliers don't really make money by effective targeting.
I witnessed this.
I briefly worked on the recommendations team for a fashion retailer, doing yeoman plumbing and maintenance. Basically an in-house equiv of RichRelevance.com.
So, so much data and logging.
Alas, after cutting out all the obfuscation layers and steps, the "lift" attributed to the team's exquisite algos proved illusory. Some >80% of the conversions came from the "recently viewed" rule. Whereas the conversation attributed to the algos consuming most of our dev and compute resources was basically noise.
I wouldn't say our (very smart) data scientists were fooling themselves, necessarily. More like the feedback loops were never fully validated. (Familiar story of non-programmer domain experts hoisting a POC, which then was promoted to production. Maintenance hell.)
Coincidentally, FWIW, my bro has worked in ad tech (Avenue A/Microsoft, RichRelevance, others?). He's got a math skillset. Was a true believer. It took a long time, but he eventually lost his "faith" that digital advertising (as they were doing it) was effective, on the balance.
Also FWIW, during my brief stint, I advocated "personalization". What I imagine StitchFix is doing. I'd love to know, for real, how they're doing. Like the split between algos and human "style consultants".
Practically all web marketing experiments and analysis is invalid, and not replicable. But it pays the bills for everyone to pretend its legitimate.
The worst part is that some of the tests were still used to justify decisions to and by management. SMH
It’s a global optimization problem, not a scientific way to understand how a specific change impacts users. Testers that have this mindset tend to be locally constrained and less likely to have bigger wins.
No. It's the fault of middle management.
If you are unable to provide your manager reasons for new development, he's going to find someone else to do your job, someone who will give him a report with charts and numbers that he can use to justify and expand his teams operational and headcount budget.
Giving a manager a report that says "there are little to no modifications that we can make at this time to improve UX" is a CLM for him.
Is it only their marketing power, or perhaps something about the platform better?
Not trying to be confrontational, just interested in finding out why.
To be honest Ebay, just like Craigslist, didn't change much in the last 20 years.
Sometimes you want one of those Chinese paper cutters, or reasonably priced leather tools. Ebay is mostly good, despite itself. They've also barely changed the UI in my memory, and I've got a 20 year old account.
[1] https://thehustle.co/01072021-uber-ad-spend/
[2] https://www.forbes.com/sites/augustinefou/2021/01/02/when-bi...
You can construct experiments to demonstrate damn near anything is true, and without (or even with) a rigorous background in statistics and scientific methodology, it's very hard to examine the supposed evidence.
I know I say this a lot, but constructing solid experiments is time consuming and difficult. It's hard even if you you're trying to be fair. Professors routinely get this wrong despite a literal decade of formal training in the scientific method and half a lifetime of practical experience with this exact craft; despite how fucking up puts their career, reputation and livelihood in jeopardy. Not only does this happen a lot, bad science routinely slips through peer review as well.
That's the state of affairs in the spaces that in general aren't extremely adversarial, where the people involved in constructing the experiments in general don't have a a vested interest in trying to mislead you. (Scientific fraud still happens of course, but Jan Hendrik Schön is the exception and not the rule)