News for hackers. Stuff that matters.
Re:What do we really know about the effectiveness of digital... by davidedicillo on Wednesday November 06, 2019 @02:48PM (#21466383)
Overstated.

This article assumes that: - All online advertising is Google search - That brand search advertising is representative of the entire industry

There are plenty of companies making double-digit ROAS (return over ad spend), especially direct response advertisers.

Re:What do we really know about the effectiveness of digital... by jessefrederik on Wednesday November 06, 2019 @03:47PM (#21467051)
Hi, I'm the author of the article, and I don't assume all online advertising is Google search. I also cite research on display advertising (meta-study of 432 display experiments on Google): https://www.ssrn.com/abstract=2701578 And on Facebook advertising: https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...

The problem with all online advertising is that there are huge selection effects, which are hard to correct for using conventional statistical methods. So you need to do experiments. And when economists do experiment, they find that advertising effects are so small that they are hard to measure.

Re:What do we really know about the effectiveness of digital... by marrone12 on Wednesday November 06, 2019 @04:05PM (#21467285)
There is a selection bias in these studies as well -- that you are focusing on large, already established brands that have good brand recall and people who had intent to purchase there anyways.

For smaller companies that are just starting out, it's relatively impossible to have prior intent when people don't know who you are or what you offer. There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.

Re:What do we really know about the effectiveness of digital... by sharkmerry on Wednesday November 06, 2019 @04:30PM (#21467577)
> There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.

But is that number statistically significant? some do, how many others tried the same route and failed, how many grew without it, etc

Re:What do we really know about the effectiveness of digital... by graycat on Wednesday November 06, 2019 @04:41PM (#21467717)
> using conventional statistical methods.

Experimental design, analysis of variance are such. E.g., for the farmers and from the corn fields and hog pens of Iowa:

George W. Snedecor and William G. Cochran, Statistical Methods, The Iowa State University Press, Ames, Iowa.

These methods have been widely used in the social sciences -- e.g., my wife, Ph.D. in mathematical sociology from Hopkins, got quite good with that material. The field is quite serious and mature and goes well beyond just A/B testing.

For the practical challenges of the article, academic fields closer than economics include statistics and optimization.

For the Lagrange multipliers in the article, those likely would be from the Kuhn-Tucker (Karush-Kuhn-Tucker) conditions. There without some special assumptions, e.g., having to do with cases of convexity, the conditions are only necessary for optimality and not sufficient. Generally in practice, it is more difficult to get sufficient conditions.

Yes, correlation does not necessarily mean causality. Usually showing causality needs a mechanism; in practice showing causality just from data and/or statistical methods is difficult and rare. But in practice, correlation can be powerful enough to take money to the bank.

Re:What do we really know about the effectiveness of digital... by glofish on Wednesday November 06, 2019 @04:02PM (#21467255)
Do you think that the situation is due to an arms race kind of scenario? -If everyone but one stopped advertising then the one still doing it would reap massive benefits? So advertising is a form of obstructing the competitor.
Re:What do we really know about the effectiveness of digital... by mochomocha on Wednesday November 06, 2019 @11:43PM (#21470455)
I can confirm this. I've run similar studies at my current employer (some of them with one of the persons you cite in your article) on multiple very large advertising platforms for display advertising. The causal ad effect is very often indistinguishable from statistical noise.
Re:What do we really know about the effectiveness of digital... by neonate on Wednesday November 06, 2019 @03:05PM (#21466570)
Maybe overstated, but if "all" the article shows is that Google search ads don't work, that's a big deal.
Re:What do we really know about the effectiveness of digital... by buboard on Wednesday November 06, 2019 @05:26PM (#21468240)
Worth mentioning the conflict of interest that you work for facebook's Ads Growth.
Re:What do we really know about the effectiveness of digital... by davidedicillo on Wednesday November 06, 2019 @11:54PM (#21470519)
Ah, not anymore. Outdated information, I work on a different advertising platform :)
Re:What do we really know about the effectiveness of digital... by nabla9 on Wednesday November 06, 2019 @03:25PM (#21466785)
How do you know it's not selection effect and the ROAS number are BS just like they are with Google? Have you tested?

Google and Facebook are 60% of the market, so if it would be only them this is important.

Re:What do we really know about the effectiveness of digital... by mochomocha on Wednesday November 06, 2019 @11:50PM (#21470492)
The ROAS numbers given by FB (your employer) are far from being a real estimate of the causal effect of adverts. For example, there's no way to have global holdback against other advertising channels not controlled by FB. Double-digit anything are easy to get on any advertising perfomance metric if you don't scale your spend, so it's not really an argument.

(Disclaimer: I've built multiple bidders & ads incrementality models over the years so I'm probably biased against the "supply" side of ad inventory).

Re:What do we really know about the effectiveness of digital... by dontich on Wednesday November 06, 2019 @05:08PM (#21468037)
If a company has incrementality tested and is really making double digit ROAS they aren't spending enough haha.
Re:What do we really know about the effectiveness of digital... by mr_toad on Wednesday November 06, 2019 @06:30PM (#21468743)
Not sure if you’re joking or not, but it makes obvious economic sense to keep investing in something that has a higher rate of return than other investments - and you can’t reliably get a 50% return on any other investment I’m aware of.

And it only takes a moments thought to realise that once everyone starts doing this, the marginal rate of return is going to be driven down to the going rate in the rest of the market.

Like any other investment, if it sounds too good to be true it almost certainly is.

Re:What do we really know about the effectiveness of digital... by tomrod on Wednesday November 06, 2019 @07:31PM (#21469202)
As the other commentator identified, you work @ FB -- could you please provide the following three scatter plots to support or disprove your claims?:

(1) Size of Company vs. ROA

(2) Advertising Expenditure by Company vs. ROA

(3) [Advert %] vs. ROA

I hypothesize you'll see a rapidly decreasing return on advertising due to size, and would love to be proven wrong.

Re:What do we really know about the effectiveness of digital... by davidedicillo on Wednesday November 06, 2019 @11:59PM (#21470540)
Not at FB anymore but I would assume your hypothesis isn't too far from the truth. It's also true that there are hundreds of thousands of small businesses that thrive thanks to online advertising.
Re:What do we really know about the effectiveness of digital... by tomrod on Thursday November 07, 2019 @05:48AM (#21472084)
The US currently has about 25M small businesses. How many of them use digital advertising? What testing is done to ensure their ROA isn't a waste?