News for hackers. Stuff that matters.
Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @02:52PM (#28855895)
There are so many holes in these arguments that you could drive a truck through it. Which is so infuriating, because a big part of my professional career consisted of watching all of Rand's SEO videos and really appreciating them. I really thought he was a genius. But then over the past few years, he started sharing more general views on entrepreneurship, and those takeaways just didn't really make much sense. Basically, his own VC-funded company turned into a shit show and suddenly he started advocating against VC in general (as opposed to taking an honest look at the mistakes that his company made). So in the past few years, I tried to reconcile in my head: how can a genius make such imperfect conclusions? My initial takeaway was that he's blinded by his own mistakes and shifting the blame, which seems perfectly reasonable and understandable. Frankly, I would probably feel and act the same way.

But after reading this article, it finally dawned on me. He makes imperfect conclusions in everything he touches, it's just that in some fields those conclusions can be more easily proved to be wrong than in others. SEO is the perfect field where a polished presenter can get away with imperfect conclusions for years - trust me, I know, I made a living for years in this field, and I am very familiar with the nature of this work. Most of the time, you have no idea what the black box really does, and instead you're just trying to guess what might have happened. Most importantly, there are many ways to skin a cat in SEO, and just because your approach is net positive doesn't mean that you truly are delivering the global maximum (or that the net positive gain was ROI positive). In short, it's impossible to know who's right and who's wrong, and Rand's videos convinced me that he's right, but I am no longer sure. I just rewatched one of them, and can easily see how his conclusions are just... opinions.

While we may or may never find out if his SEO opinions were the global maximum, we can quantifiably demonstrate that his opinions on content marketing are not solid. This whole essay he wrote can be replaced with "hey performance marketers, don't trust the platform numbers and instead do your incrementality studies." Platforms like Facebook will give you those for free if you reach a certain spend level, and you can also get them from 3rd party providers like measured.com. In other words, if you're a performance marketer and you're not conducting incrementality studies, then you're very early in your career and are not following the best practices. Simple as that - no need to extrapolate from there and reach all sorts of additional conclusions (which is obviously a pattern in Rand's behavior) - calling into question a perfectly investable marketing channel, conflating the needs of a public company with everyone else's needs, using words like scam, etc.

I am really disappointed to have to write this, but you would have been better off not reading this article. If Rand is really advocating that the majority of entrepreneurs should follow his advice and focus on PR instead of performance marketing, then perhaps an honest thing to ask would be - how is that working out for his own company? AFAIK, SparkToro is nowhere close to replicating the growth of his previous company, which is honestly disappointing for someone with such a huge reach and name recognition.

Re:What if performance advertising is just an analytics scam? by ChuckMcM on Wednesday October 13, 2021 @02:55PM (#28855947)
Can I politely suggest that you share with us the holes you see in the arguments and how you see them? Whether or not Rand is any good at what he does or says will be clear by your expert disassembly of his arguments.
Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @03:38PM (#28856557)
- Brian's comments about Airbnb's approach to performance marketing are used to imply that Airbnb's lessons apply to other businesses. Very few businesses - especially those that start from small numbers and need to grow them - have 90% of their traffic mix come from repeat customers. Using Brian's comments helps fuel the narrative that brand marketing is a better investment than performance marketing, which is correct in some cases (Nike, Airbnb), and not in others (most startups). Also, as another member pointed out, Airbnb's performance marketing budget is still well over $200M/year, which no responsible/public company would spend if it wasn't returning a great ROAS. Finally, Airbnb is known for all sorts of marketing shenanigans in their early days, and they certainly can't take the credit for a pure brand play.

- Calling performance marketing platforms a scam (repeatedly, both in the title and in the narrative) doesn't explain how those same performance marketing platforms are carrying the majority of traffic acquisition in most of the B2C companies that went public this year (and practically all of the DTC ones). Calling into question the accuracy of measurement is one thing. Calling it a scam is wrong and designed to rank on HN rather than to be reflective of the true value of those platforms.

- As I pointed out in my original post, all you have to do is use incrementality studies and 98% of the criticism instantly goes away. Rand implies that you have to do your own studies (by eg, following Avinash Kaushik's methodology) which is 100% wrong - Facebook will do them for you if you reach a certain spend limit, or 3rd parties will as well with no spend limits. Also, from experience, this really becomes an issue once you spend meaningful amounts on two platforms at the same time. His rant on this subject has an iota of truth and a whole lot of sensationalism mixed together, and overall leads to wrong conslusions.

- He conflates "paid search" with "all performance marketing platforms", including "paid social." It would have been helpful to point out that the challenges with branded terms are entirely isolated to paid search and have nothing to do with paid social.

- My favorite sensationalist tactic: frame a strong accusation as a question. This way you get the clicks, but you can still cover your ass by linking to resources that with enough research would allow the reader to answer the question with a "No." But in lieu of that research, the implication is that the answer is a "Yes." You'll see this tactic used by less reputative media sources, and I was disappointed to see Rand do the same.

I could go on but hopefully this will suffice.

Re:What if performance advertising is just an analytics scam? by inthewoods on Wednesday October 13, 2021 @06:54PM (#28858585)
"Also, as another member pointed out, Airbnb's performance marketing budget is still well over $200M/year, which no responsible/public company would spend if it wasn't returning a great ROAS. Finally, Airbnb is known for all sorts of marketing shenanigans in their early days, and they certainly can't take the credit for a pure brand play."

My experience is exactly the opposite. The larger the budget, the less real hard analysis is done. This is especially true with the rise of attribution modeling which allows marketers to essentially motion blur the data.

Re:What if performance advertising is just an analytics scam? by aerosmile on Thursday October 14, 2021 @12:24AM (#28860505)
The larger the budget, the more sophisticated the teams and the methodology. Also, more players eager to accuse marketing of poor performance (eg: sales and revenue teams). Not sure where you made that experience, but that marketing team would get fired within a quarter at any large scale company I am familiar with. Large companies are inefficient in many ways, but not in what pays the bills.
Re:What if performance advertising is just an analytics scam? by inthewoods on Thursday October 14, 2021 @05:09AM (#28861991)
Many of the large companies will say "we have a complex sales cycle" (and they often do) and leverage multi-touch attribution models which removes the direct ROI calculation.
Re:What if performance advertising is just an analytics scam? by lostinquebec on Wednesday October 13, 2021 @04:17PM (#28857139)
Your first point is very hopeful, but not what I've experienced. The bigger the company, the more slack for bad decisions.

Your other points I think relate to scale. No advice can be universal, and if you read the article as absolutist, your take makes sense. If you read it as "hey, your mix is likely wrong", a lot of the criticism fades.

I think we've lost a bit of creativity in marketing. The Lego movie example is a really good one. I think it is probably good this happened, as a lot of creativity was performative (how do I win an ad award/impress my peers) and not about increasing sales, but we've perhaps shifted the balance too far, and there is likely some areas with good ROS that are now better bets.

Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @04:32PM (#28857312)
Scale brings both, headwinds and tailwinds. The outcome often hinges on how the balance of those two forces plays out.

Ecommerce example: every year the CPMs go up and your paid margin goes down. But every year you have a larger email list, so the balance of paid to unpaid shifts.

General example: every year you get more of the late-stage employees who care less and less about your company. But every year you can afford to pay for more layers of management, which will keep an eye on the underperformers.

This list goes on and on... The headwinds are driven by external forces, whereas the tailwinds end up working out based on your specific execution of the opportunities that present themselves to you. This is where an experienced operational team can make a huge difference.

> I think we've lost a bit of creativity in marketing.

You absolutely cannot rely on performance marketing forever. It's a shot in the arm until you have reached enough [fill in whatever you wnat] so that you can leverage that momentum to reach the escape velocity. So it's not good forever, but it's a great catalyst.

Re:What if performance advertising is just an analytics scam? by NumberCruncher on Wednesday October 13, 2021 @04:32PM (#28857305)
> Rand implies that you have to do your own studies [...] which is 100% wrong - Facebook will do them for you

So FB, who earns on my spending, offers me to measure for me whether my spending makes sense for me. Why would FB ever tell me to spend less? Don't they like money?

Re:What if performance advertising is just an analytics scam? by disgruntledphd2 on Thursday October 14, 2021 @05:44AM (#28862169)
> Why would FB ever tell me to spend less?

There's literally a team at Facebook (Marketing Science), part of who's job is to tell advertisers what actually works and often, this can lead to them telling advertisers to spend less.

And this is an incredibly smart business strategy in that FB know (from experiments et al) that their platform works, and if they can show incrementality, then advertisers will invest more in the platform.

In game theory terms, it's a good strategy for a repeated game, which advertising definitely is.

Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @04:35PM (#28857339)
It's fair to question credibility, but you have to realize that their results can be compared against 3rd parties, and it would lead to a lot of reputational damage (and also legal risk) to systematically defraud your advertisers. All it takes is one smart team to out the entire enterprise.
Re:What if performance advertising is just an analytics scam? by stoicking on Wednesday October 13, 2021 @06:04PM (#28858233)
The causal incrementality of ad spend is rarely negative. You expect to see decreasing returns with additional ad spend. For each dollar in ad spend, one advertiser might want to see $3 in revenue, another $5. Incrementality studies then allow advertisers to tune spend to their operating points.

Facebook does like money. Do you think lying to the biggest advertisers in the world is the best long-term strategy, or do you think instead it might be better to report out the most accurate results possible?

Re:What if performance advertising is just an analytics scam? by lmm on Wednesday October 13, 2021 @10:45PM (#28859989)
Facebook has a history of screwing advertisers ("want your page to show up in people's feeds? Get more follows, here's how to buy ads targeting that", then one year later "we've decided to downgrade pages in people's feeds"...), bankers (their IPO pricing), and essentially everyone they have a business relationship with. I'd expect them to stretch the truth as far as they can get away with.
Re:What if performance advertising is just an analytics scam? by Androider on Wednesday October 13, 2021 @07:17PM (#28858731)
Re:What if performance advertising is just an analytics scam? by dsizzle on Wednesday October 13, 2021 @05:26PM (#28857920)
To your point that AirBnB's and other global brands don't extend to most businesses, Facebook's recent outage provided such an experiment and indeed a lot of small businesses saw massive effects https://mashable.com/article/facebook-outage-small-business-...
Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @06:32PM (#28858435)
Thank you for sharing this. Facebook received a lot of criticism for how they lobbied publicly against iOS14. Their reputation is ruined, and anything they say will always be taking with a cynical afterthought. But as much as it pains me to say it, they are 100% right in saying that iOS14 is indirectly going to create a lot of damage for SMBs and companies trying to disrupt the status quo. There's more nuance to it, but think of it this way: if your marketing budget plans on reaching 100m Americans each year, then the negative impact on micro targeting is not quite as bad since you're going for the mainstream customer anyway. But if you're trying to reach only 1m people, then losing micro targeting is a matter of life or death.
Re:What if performance advertising is just an analytics scam? by croes on Wednesday October 13, 2021 @08:46PM (#28859308)
>Also, as another member pointed out, Airbnb's performance marketing budget is still well over $200M/year, which no responsible/public company would spend if it wasn't returning a great ROAS

That's not really an argument. You wish that a responsible/public company wouldn't do that but we have seen much dumber behavior.

Re:What if performance advertising is just an analytics scam? by aerosmile on Thursday October 14, 2021 @12:18AM (#28860478)
Now I am curious - what are the examples of unprofitable performance marketing campaigns at that scale conducted by public companies?
Re:What if performance advertising is just an analytics scam? by dangrossman on Thursday October 14, 2021 @02:17AM (#28861071)
> When P&G turned off $200 million of their digital ad spending, they saw NO CHANGE in business outcomes. When Chase reduced their programmatic reach from 400,000 sites showing its ads to 5,000 sites (a 99% decrease), they saw NO CHANGE in business outcomes. When Uber turned off $120 million of their digital ad spending meant to drive more app installs, they saw NO CHANGE in the rate of app installs.

https://www.forbes.com/sites/augustinefou/2021/01/02/when-bi...

Re:What if performance advertising is just an analytics scam? by aerosmile on Thursday October 14, 2021 @07:05AM (#28862743)
That was an incredibly misleading article. If you check the links, you find the following:

1. P&G didn't turn off that spend, it shifted it to other marketing platforms.

2. Chase didn't change its marketing spend, it just concentrated it from 400k sites to 5k sites.

3. Uber found out that their agency was committing fraud and AFAIK the case is still being litigated.

I hate being that guy, but check the links. Those reporters really must have an agenda or something (or are just struggling to get the clicks, so they need to make a story out of nothing).

Re:What if performance advertising is just an analytics scam? by croes on Thursday October 14, 2021 @11:18AM (#28865481)
To 1. "P&G’s $200 million digital cut were reinvested into areas with “media reach” including television, audio and ecommerce" So they awitched to other platforms with mostly no back channel and no user tracking. So the whole tracking seems to be pointless
Re:What if performance advertising is just an analytics scam? by disgruntledphd2 on Thursday October 14, 2021 @05:46AM (#28862184)
Like, this is probably true for advertisers who have a well known brand. One would expect incrementality to be lower, because everyone's already heard of the brand.

However, if you are a new entrant/small business this is not the case, because nobody has ever heard of you.

As an example, look at TikTok. They spent insane amounts of money on FB ads to get as many installs as possible, but I'd suspect that they don't do this anymore because they've got enough brand equity that it doesn't make as much financial sense.

Re:What if performance advertising is just an analytics scam? by croes on Thursday October 14, 2021 @05:00AM (#28861953)
My point is not exclusively advertising but that it is wishful thinking that companies always make sensible decisions. Often enough, useless or even harmful things continue to be pushed through, simply so that one doesn't have to admit to oneself that one has made a mistake. They would rather try to achieve a positive result with larger investments after all, which usually fails and leads to even larger losses, than to exit early and get away with smaller losses. Therefore, the statement that a reasonable company would not act this way is not covered by reality. Companies should not act this way, but they do.

https://en.m.wikipedia.org/wiki/Escalation_of_commitment

Re:What if performance advertising is just an analytics scam? by engineeringwoke on Wednesday October 13, 2021 @03:23PM (#28856324)
People on this board love the idea that anyone who isn't an engineer is an idiot and everything that everyone else does that isn't writing code is somehow worthless. In broad strokes, they think that all people who do people-facing work are charlatans that are constantly trying to hide how worthless they are because talking to people, etc. can't possibly provide value. Then, they (like so), ask you to break down your arguments so they can pin the tail on the donkey that is the logical fallacy that they will use to discount your (almost certainly correct and informed) opinion.

Unsurprisingly, those people are wrong and marketing works.

Re:What if performance advertising is just an analytics scam? by photochemsyn on Wednesday October 13, 2021 @04:35PM (#28857341)
It's also fair to say that if you let the marketers drive business decisions it's a recipe for disaster. Pushing low-quality product out the door just to hit some quarterly sales goal, that's what marketers will push. Epic failures with disastrous long-term consequences are the typical result (Boeing MAX for example).
Re:What if performance advertising is just an analytics scam? by lukestevens on Wednesday October 13, 2021 @08:04PM (#28859039)
If you think the Boing 737 MAX disaster was caused by those idiots in marketing calling the shots then you just proved engineeringwoke’s point.
Re:What if performance advertising is just an analytics scam? by photochemsyn on Thursday October 14, 2021 @04:38PM (#28869735)
My understanding was that Boeing wanted to prevent Airbus from winning contracts and so pushed the schedule hard. That's the sales and marketing division talking isn't it?
Re:What if performance advertising is just an analytics scam? by aerosmile on Thursday October 14, 2021 @12:26AM (#28860515)
This comment is pure gold
Re:What if performance advertising is just an analytics scam? by rchaud on Thursday October 14, 2021 @12:48PM (#28866686)
Are you saying Boeing's engineering problems could have been avoided if they cut their ad budget?
Re:What if performance advertising is just an analytics scam? by poetaster on Wednesday October 13, 2021 @04:47PM (#28857518)
Hmmm. Marketing works!? Well having been a sucessfull .com era ad co. founder, I suceeded in raking in the cash of the uniniated (automobile, etc, so no tears) only to see companies literally obliterated by marketing. TV was then still 20 times the spend. Still, marketing had 0 clue. Fun to drink with at the theatre, sure. I was CTO, so, shrug? No, I was incensed and tried to talk other managers client side out of giving ME money. I know some clever people in marketing, but when they smell budget, it's a movie shoot in the sahara time.

I do think things are better in some sense today. But it is a hungry beast, marketing. Oh, and lies and statistics.

Re:What if performance advertising is just an analytics scam? by bliteben on Wednesday October 13, 2021 @03:11PM (#28856159)
you don't even have to be polite can't believe I read that whole comment
Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @06:40PM (#28858480)
Rereading the post, I agree it came across as more personal than it should have been. I wish I was able to frame it more as a response to all the people advocating for alternative marketing ideas which frankly never work out. Instead, I brought in his own company into the narrative, which was below the belt. I know first hand how hard it is to get something to work, and how unlikely it is to happen time after time. I wish I could edit the comment, but HN won't let me at this point.
Re:What if performance advertising is just an analytics scam? by beervirus on Wednesday October 13, 2021 @02:58PM (#28855986)
That was an awful lot of words. Too bad none of them had anything to do with what you think this article gets wrong.
Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @03:51PM (#28856767)
It would have made the post even longer. But I corrected that in the child post. Hopefully that clarifies it.
Re:What if performance advertising is just an analytics scam? by bborud on Thursday October 14, 2021 @08:45AM (#28863504)
I worked on several web search engines in the 2000s. During that time I ended up next to someone who was in the SEO business a handful of times at dinners or meetings. When they explained what they were doing to me I kept thinking "this kind of explains how belief systems and religions are formed".

You would have these very complete theories of how search engine ranking worked - completely disconnected from reality and unburdened by any actual knowledge. If there was ever a cargo cult, SEO was it.

I still cringe when people say that they have "done SEO" on their site while I'm in the room. It is kind of like telling their mechanic they've sacrificed a goat so their car will get better mileage.

Re:What if performance advertising is just an analytics scam? by rchaud on Thursday October 14, 2021 @12:45PM (#28866645)
What's wrong with saying someone's "done SEO" if everyone at the table has the context for what that entails?

Content strategy, keyword research, Search Console integration, site speed improvements, local search business profiles -- all of these contribute to SEO. It's unwieldy to talk about them all, so sometimes people can use a shorthand.

Re:What if performance advertising is just an analytics scam? by bborud on Friday October 15, 2021 @09:58AM (#28877737)
I cringe because usually they will be talking about things that come out of what can only be called superstition and they tend to think that's it. Meanwhile, I would be sitting there and knowing a great deal about exactly what goes on when whatever web search engine i was working for at the time actually did, and not being allowed to discuss any of it.

I'd always give the same advice. Don't try to outsmart the search engine, focus on quality and consistency over time. That will always reward you in the long run. Try to game the search engine and you will find yourself in a weapons race you can't sustain over time. If you find exploitable tactics to get better ranking than you deserve, sooner or later you'll probably be pushed down the ranking. And this is really important if you're successful enough to have something to lose.

Some of what people label SEO is just "make good web sites", and that indeed works. But some of it is just pure rubbish. And making a good website isn't just "doing SEO" - it is making a good website. And a lot of people don't.

Which is why I can't take people seriously when they present it as some singular task they have completed.

Re:What if performance advertising is just an analytics scam? by lifeisstillgood on Wednesday October 13, 2021 @03:12PM (#28856166)
Incrementality studies: https://www.adroll.com/blog/marketing-analytics/beginners-gu...

Interesting article

1. "industry" is moving away from last touch attribution (pretty much what Rand complains about)

2. "Incrementality strives to identify the causal event of a conversion,"

The way to identify the causal action seems to be :

errr ... that's difficult

Re:What if performance advertising is just an analytics scam? by aerosmile on Wednesday October 13, 2021 @03:50PM (#28856742)
You linked to an article that offers no guidance on how to actually do it, which is more indicative of the article itself than of the methodology. One of the vendors in this space has a good amount of content on this topic, which you can find at the bottom of this page: https://www.measured.com/. Dislaimer: I have nothing to do with this vendor or the industry in general (apart from successfully spending money on performance channels).
Re:What if performance advertising is just an analytics scam? by lifeisstillgood on Wednesday October 13, 2021 @06:55PM (#28858594)
My apologies I should have prefaced it with "for others confused as to what incrementality is all I could find was ..."

It's a surprise to see a vendor have a clear explanation on their site however - and it's worth quoting i think

>>> Incrementality testing creates an experiment that systematically withholds media channel exposure to a representative subset of users (the control group) while maintaining normal media channel exposure to the broader user set (the test group). If the control group is both sizable enough to be statistically significant and selected at random such that they are broadly representative of the user base, then the media channel’s incremental contribution can be determined by the difference in business outcome (conversion, revenue, profitability, etc.) between the test and control groups.

Re:What if performance advertising is just an analytics scam? by 135792468 on Thursday October 14, 2021 @08:04AM (#28863170)
Don’t know who you are but your path and journey in the industry sounds exactly like mine.