“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Nico Neumann, assistant professor and fellow, Centre for Business Analytics at Melbourne Business School.
Two recent episodes of the well-known Freakonomics podcast reviewed novel academic work on advertising effectiveness under the title “Does Advertising Really Work”? A centrepiece of the podcasts was a new study by three Chicago professors – Brad Shapiro, Anna Tuchman and Günter J. Hitsch – on the role of TV advertising for consumer packaged goods (CPG). CPG, which include products such as beers, yogurts, and snacks, have traditionally been some of the main spenders on advertising.
That study provided two thought-provoking conclusions. First, the authors show that TV effectiveness is often not measured correctly and that marketers likely have hidden many negative results over decades. This idea is certainly not unrealistic – no one likes bragging about campaigns that did not work.
However, this first finding leads us to the second, more critical result. If we don’t exclude poor campaigns from our analysis and calculate the ROI using more appropriate methods, then TV is about 15-20 times less effective than many brands may have believed before. In short: Most brands seem to overinvest in TV ads.
Not surprisingly, these findings triggered a heated discussion in the advertising world. Many people raised concerns about the study. Can we trust research done by people outside of advertising? How do the results fare in comparison to other existing studies? Do the results only apply to TV ads? And what are the lessons for brands?
What makes research trustworthy?
Let’s keep in mind that there is a quality difference between ‘research’ and ‘research’. First, the Chicago study outlines its math and research approach in its openly available paper. Unfortunately, many other studies lack this relevant information, which should include assumptions, formulas underlying the models and robustness checks. Yes, these can be boring, but they matter a lot as analyzing advertising effectiveness is a complex, statistical question.
Second, the referenced research was conducted by econometricians and professors from some of the best universities in the world. They have no vested interest as opposed to “studies” commissioned by an advertising group or publisher, which somehow regularly find that their medium performs the best.
Economists are neutral because they have nothing to lose from the results and only care about the bigger picture – what is good for society?
We should all welcome such efforts. Let’s not forget that the tobacco industry also has a long history of presenting “science” that cigarettes do not pose any danger to people. When billions of dollars are at stake, we can never rule out concerns that such a study was done with a bias, which is why it’s so important to have independent investigations.
The devil is in the details
While the overall message of the Chicago study may initially seem daunting, we need to pay attention to some caveats here. For example, the researchers find that at least 11.2% of the 288 brands they looked at have successful and significant long-term ROI. They also write that more than two thirds of brands have, at some point, positive ROI in their calculations. So success varies with the brand and the individual campaign.
We should also consider whether advertising may provide other benefits that the research did not focus on. For example, advertising could reduce price sensitivity and allow brands to charge a higher premium. And of course, the presented research – while of high quality and professionally conducted – has limitations too, as every research study has. Therefore, let’s review other independent studies to get a balanced overview.
What do other independent studies say about advertising?
Other independent studies show that the Chicago study was not the first shedding light on the unfavorable economics of advertising for many brands. In the 80s, Magid Abraham and Leonard Lodish conducted a large study and found in 360 tests that increased TV advertising led to more sales only about half the time. The two authors further warned back in 1990 that senior managers needed to “throw out much of the conventional wisdom about advertising” and stated that “many companies could reduce their total advertising and promotion budgets and improve profitability.”
That ad campaigns often don’t produce positive ROI is not unique to TV advertising. Another study surveyed over 1,100 consumer products executives across six countries in 2018. Based on the survey responses by the brands’ senior managers, more than half of digital ad spend had either a negative return or its return was not even measured.
Creative work bears risks
At the beginning of the 20th century, retailer John Wanamaker said: “I know that half my advertising is wasted; the trouble is, I don’t know which half.” Fast forward 100 years and that old wisdom still applies.
Considering the nature of producing advertising, the mixed success rates of today’s brands’ efforts should not be a surprise. At its core, advertising is a creative business. We produce short films, audio pieces or other graphics and messages to capture people’s attention. Constantly succeeding with a creative business is a big challenge other creative industries face too. Many Hollywood movies create losses for their studios. Likewise, only few produced songs become hits and generate profits for music labels or artists.
Given the natural risk of all creative work, it is therefore even more crucial to measure outcomes properly. Only then can we study the effectiveness of those unique creatives and ad campaigns that stand out from the mass and deliver strong returns. And the safest way to do so is using the appropriate analytics tools (in particular experiments) and independent validation.
Maybe if more brands conduct their own tests cross-checked by experts without conflicts of interests, then we will be able to reduce the amount of wasted ad spend to less than 50% in the next 100 years.
