This first appeared in the AdNews February edition. Subscribe here for your copy.
When IAG’s media director Willem Paling switched the insurer from attribution modelling to experiment testing, he found the company needed to invest less in online advertising and more in traditional media. Nicole McInnes at WW, formerly Weight Watchers, was told the same thing when she adopted a more holistic model as the brand’s new marketing director. Each brands’ focus on attribution modelling meant digital advertising’s success was being inflated when looking at sales, while long-term brand building, usually done by traditional platforms, was overlooked simply because it was harder to measure.
The flaws in attribution modelling have long been raised throughout the industry, yet its use remains widespread. Figures on its adoption vary, but according to a report sponsored by Google Analytics, released in 2012, 62% of marketers and 77% of agencies used attribution. This was from 607 responses around the globe. The report also found that last-click attribution was the most popular model, used by 54% of agencies and 54% of clients. The reason; because it was the most “readily available” model.
A joint AdNews and AdRoll 2016 study found that 66% of Australian marketers were using some form of attribution. This jumped to 90% the following year, according to a separate study by AdRoll. The study also found first-click and last-click were the most popular, at 44% and 28%, respectively.
“Everyone is happy with fake attribution,” says Melbourne Business School professor Nico Neumann speaking to AdNews about the current state of attribution modelling. The professor repeatedly labels the practice as “fake”. Like many critics, he rejects that attribution modelling offers valuable measurements for brands or marketers because rather than showing how effective advertising is by looking at incremental sales, it claims full credit of sales even in many cases when it shouldn’t claim any.
“We’ve published about this for years, that the rules don’t make any sense,” Nuemann says.
“But still, I would guess that a high percentage, probably 50%-80%, still use basic rules.”
Touch-based attribution models work by attributing a sale to an ad that’s been delivered to a person before their purchase. This is regardless of whether the person saw the ad or not – and many studies show a majority of digital ads are never seen. For example, ads that appear at the bottom of pages that a user never scrolls down to. The problem doesn’t stop there. The Association of National Advertisers found that only one-quarter of all digital ad spend reaches actual people.
Click-based attribution, on the other hand, uses the last ad a user clicked on, a slightly better model, according to some experts because it reflects an action taken by users.
Variations of touch and click-based models, such as linear and time-decay, simply attribute the sale by different weightings. While first-click attribution gives the first click 100% credit for the sale, linear splits it between the first and last click, and so on. The ads attributed can be days or weeks old.
“That is the stupidest thing ever because it suggests all your sales are only driven by advertising,” Neumann says.
“You can’t just take $100 of your sales and say this was driven by advertising. The whole idea, and this is what the standard attribution tools do, basically claims that every sale you make is because of advertising and that’s not true.”
Simply, correlation doesn’t always equal causation.
WW’s Nicole McInnes is also wary about the way attribution inflates the role of advertising, particularly digital, in sales and describes attribution as being in the “Digital Middle Ages”. When she joined the lifestyle brand last year with years of experience working around attribution, she adopted a more holistic approach to measure advertising and marketing results.
“Attribution was meant to deliver this beautiful pathway to the perfect media mix, a sure uplift in sales and of course, lots of growth,” McInnes says.
“In fact, what it ended up delivering was the opposite, because it influenced marketers to move all of their media into channels that could be tracked. Unfortunately, easily trackable channels are mostly mid and lower-funnel channels.
“And so they basically shrunk their potential audience by reducing their share of voice and stunted the growth of their companies. Year-on-year declines started to appear after one to two years, from some examples I heard about.
“The problem was compounded by two factors, one; the short term results were good, and made marketers look and feel good; and two; board executives got wind of this ‘silver bullet’ and rewarded anything and anyone ploughing money into digital, while questioning anyone who didn’t.
“I love digital, and have worked now in four digital native companies, but it is important to understand the role of every channel, highly measurable or not, and also brand salience and the need for customers to be emotionally impacted by your campaign to even remember your brand, let alone purchase it."
Prior to the changes, WW was working with an agency that would often recommend spending more on digital. After investing in media mix modelling software that took in offline channel spend and sales, it revealed an underspend in TV and radio, and an overspend on things such as generic search and display.
Nicole McInnes
Big brands, such as Adidas, have been coming out publicly admitting they also over invested in digital advertising. The sporting brand’s global media director Simon Peel told media the skew towards digital was due to a number of factors, including reliance on last-click attribution, a lack of econometric modelling and the failure to do any brand tracking.
“I’m starting to see the light going on for other people,” McInnes says.
“People are realising channels such as TV and outdoor are still very effective in the media mix, despite them not being easy to get a clear return on figure on.
“Until attribution modelling becomes more sophisticated, marketers can’t really know how effective a channel is until it’s turned off through classic A/B testing.”
Back to basics: experiment testing and econometric modelling
Neumann advocates for a return to running experiments to test the effectiveness of advertising. He argues that through this approach, advertising will be seen as how much extra value it provides a brand, rather than what short-term sales it results in. This is what marketers should really be trying to identify, Neumann says.
“Marketers should be asking ‘what is the difference the ad made to you? Did I show you an ad and did it inspire you at some point to maybe make a purchase?’,” Neumann says.
“Attribution models, as most companies use them, don’t really do that. They’re just some random math to put some numbers next to the advertising they use.”
To run the experiments, one group is allocated to see ads, while another isn’t exposed to the ad and sales figures between the two groups are compared.
“The sad thing is that companies would be much better just running experiments,” he says.
“The tricky part of experiments is to just plan them and set them up. Once they are set up correctly, everyone can understand the research. You don’t need a math degree to understand it.”
Four years ago, Paling, now IAG’s customer and growth analytics director, dumped attribution modelling at the insurer in favour of running experiments.
Like Neumann, he tells AdNews digital attribution models aren’t good measures of the effectiveness of advertising, as attribution, particularly impression-based attribution, can easily be optimised towards touching people with non-viewable impressions who were going to buy a brand’s products anyway.
For a small subset of marketing, such as generic search, Paling says it’s not a stretch to say that someone searching for car insurance, clicking on an ad for car insurance and then making the purchase, did so because of that ad.
However, for marketing that is targeted to a brand-specific purchase intent, such as branded search or display retargeting, it is a stretch, he says, to say that just because a person saw the ad, it means that a sale happened that otherwise wouldn’t have happened.
IAG’s experiments now run out of its Customer Labs division, guided by an attribution framework developed with the help of physicists to isolate the causal impact of marketing. The number of people working within the team changes depending on the number of experiments it wants to run.
“The physics comparison is really interesting because physicists are quite good at dealing with systems which are not very controlled,” Paling says.
“There’s a lot going on in understanding the movement of a star, or understanding what’s going on in the Hadron Collider. Marketing is kind of similar in that your ads going into the market are a very small part of a very complex market. So isolating the impact of advertising is a very difficult, scientific problem.”
The more experiments it ran, IAG, owner of NRMA, found that the majority of channels weren’t delivering real short-term uplift in sales.
It cut down on branded search, after finding it delivered little uplift to sales, as well as retargeting both because it doesn’t deliver a sales uplift, and it’s something that “annoys” customers.
During an experiment it ran for 19 weeks, last-touch attribution showed retargeting delivered about 1,800 sales and an algorithmic attribution model showed it delivered about 1,700 sales. However, the experiments it ran found that between the groups shown ads and those not shown ads, there was a difference of just 26 sales.
“We found generic search, where people are searching for our brand, looks amazing through any digital attribution model because people are saying, 'Hey, I want to buy NRMA car insurance'. We put an ad in front of them, and they still want to buy NRMA car insurance.
“But intuitively, you know that most of those people were going to buy NRMA car insurance anyway.”
Overall, from 2017-18, 30% of digital spend was proven to be delivering no return, and moved into broadcast media by IAG.
“In general, there are a lot of performance channels that do really well at last-touch attribution, like native advertising and social and performance display. And so this demonstrated to us that these only did well because they did well at gaming last-touch attribution, not because they were actually causing an increase in sales.”
Paling says this is likely unique to items that aren’t impulse buys, such as insurance, compared to other categories, such as clothing and food.
“More ads don’t generally cause people to respond by going out and buying insurance. In general, people come to that decision when they have the need, and then they think of just a handful of brands, often only one,” he says.
“And so the ads are doing a long-term job to create the mental availability that means when they come to that insurance decision, they think of you. And that’s much, much harder to measure.”
There are three challenges for brands looking to run experiments, one being opportunity cost – the cost of turning off advertising in some markets, which can be scary for some.
McInnes, who has conducted “dark weeks” in a previous role at EHarmony, says it can be harder for marketers in larger organisations to do this, as they risk losing sales.
“In the end when I have to choose between knowing with statistical significance and growing, I’ll go with growth every day,” she says. “That is until the two align, which I am hopeful they one day will.”
Another challenge, Neumann says, is that experiments can be time consuming.
“But to be honest with you, the biggest challenge I have found, is people don’t like what they find,” Neumann says.
“As soon as you move away from what I call the pseudoscience and fake attribution, from the unrealistic measures, suddenly your advertising doesn’t look that good anymore.”
Neumann claims switching to experiments can drop the success rate of retargeting from 70% of sales down to 5%.
“In reality, it’s not that you have 70% or 80% of your sales driven by advertising. That’s just not true,” he says.
“But here’s the thing, everyone is happy with fake attribution.
“The ad agency is happy because advertising makes them look good. The marketers have a conflict of interest with agencies who are responsible for buying the advertising, because it shows they did a good job.”
He argues marketing departments shouldn’t analyse their own work because they have the same interest – to ensure the results match the costs.
“Everyone wants to do that in their job,” Nuemann says. “They get away with it because the excuse is that everyone else is using attribution modelling. If you suddenly change it to a real scientific way, everyone is scared of what it may mean for their job, what it means to their budget, and everyone is rather happy to pretend the wrong thing is true.”
IAG’s Paling echoes Neumann’s concerns, saying it was difficult to manage all the different players; tech companies, publishers, adtech companies; during his transition to experiments, as they were all used to using attribution modelling to claim uplifts in sales.
Most demand-side platforms (DSPs) are optimised towards last-interactions attribution, generally last-impression rather than click, because not many people click on display ads and to develop a more robust approach would mean attributed sales would go down “drastically”, Paling says.
“For everyone, it’s a huge overhaul to say the amount of sales that we’ve been claiming is inflated by 10 times or more,” he says.
“The only way it can happen is that people start to see that they’re delivering less sales than they should and it’ll come about that display advertising behaves in the same way as TV, radio, outdoor and so on.
“For most categories and products, it’s more of a brand building activity than it is a cause for someone to change their behaviour and buy it when they otherwise wouldn’t have bought.”
Paling puts the number of display ads that are viewable at 40%. Other studies back this figure, with one by Solve Media saying people are more likely to survive a plane crash than click on a display ad.
“So there’s this ridiculous practice going on where most DSPs are optimising towards putting a non-viewable impression in front of someone who was going to buy the product anyway,” Paling says.
“I don’t know why more marketers aren’t waking up to it and saying, ‘this is ridiculous, we’re not going to do it’.
“There’s this kind of suspension of disbelief. I think marketers know that it’s not true that they’re getting these sales. But internally if you’re a marketing manager, to make a switch from a last-interaction attribution model to a scientific attribution model, the number of sales that are happening as a result of your activity and the perception of what you’re delivering to the business is going to decrease maybe 10 fold.
“So it’s a very brave thing for a marketer to say, ‘I’m going to switch to show you the true value of what I’m delivering’, because they know that short-term reported value of what they’re delivering is going to go down.”
Peter Danaher, professor of marketing at Monash University, agrees attribution is used because it is intuitive, not because it is valuable. He says while there’s interest in experiments, many companies are nervous about the “damaging” effects on sales when ads are halted for sample customers. Instead he thinks econometric modelling is a better approach. This incorporates a range of factors that contribute to a sale, such as offline and online media, price, promotion and distribution.
While this method is harder to do, Danaher argues the availability of “single-source data”, where you have each person’s exposures to advertising and purchase history, means it’s more effective.
“Nowadays it’s much more common for companies, especially through loyalty programs, to have people’s advertising exposure information and their purchase history information. And when you have that you can put together essentially a mathematical model which links advertising exposures to purchase,” Danaher says.
“It’s more difficult to do, but it actually gets to what you’re trying to figure out, which is the relative importance of the different media.”
Danaher has previously worked with brands such as Kmart and Myer to implement econometric modelling, but says there isn’t “enormous popularity” in the area.
“But I think smarter advertisers have worked out that there are better ways to allocate your money than through attribution.”
Rebalancing skewed spending
Danaher, alongside Harald Heerde, recently released a study called Delusion in Attribution: Caveats in Using Attribution for Multimedia Budget Allocation. It argues attribution modelling is unjustly skewing advertising spend.
“There’s a lot of herd instinct with regard to media,” Danaher says.
“Every time I pick up a new survey, it doesn't matter which country you’re in, you see money going out of traditional media and more into digital media."
While Danaher thinks tech companies such as Facebook market their ability to target users with ads well, he warns online media has a few flaws, such as fraud and viewability issues, which are being “swept under the carpet”.
“I’m not saying traditional media is perfect either, but I wouldn’t be too hasty to toss out the baby with the bath water,” Danaher says.
His paper found that traditional media, particularly catalogues, have a lot of advertising carryover, while emails and paid search tend to be “very short-lived”. Because of this, catalogues do much better on an econometric model than they do in attribution models.
Danaher argues that a shift from attribution modelling, to either experiments or econometric modelling, will result in changes to how media budgets are spent.
While the current sluggish market is impacted by wider economic conditions, this change to spending could help struggling traditional media owners. October figures showed a 14th consecutive month of negative growth, at 8.5%, for all major media, according to media agency booking numbers.
Danaher says traditional media owners could be doing more to improve the understanding around alternatives to attribution models, and therefore the effectiveness of advertising on their platforms.
“They’re in a bit of a dog fight here,” he says.
“I mean really their currency is being eroded by digital media. I’m surprised they don’t react to that more strongly than they do because it’s kind of at their core – advertising revenue is the basis on which they survive.
“They need to fight back and do studies like this sort of thing I’ve done, which should demonstrate that traditional media isn’t an old hat, that it can be really, really effective.”
Media owners, from Nine to Southern Cross Austereo, recently came together to do just this in a campaign called Advertise or Die. The push, launched in December, promotes advertising on their platforms to the c-suite. It’s been described as the biggest joint effort from Australian media and aims to encourage CEOs and CFOs to advertise on traditional platforms, particularly in tough economic conditions, for long-term brand-building.
Liana Dubois, Nine’s director of Powered, says the increased adoption of digital attribution is one factor behind the shift from traditional to digital media. She says it’s become a “deadly drug for brands” largely because of pressures to boost sales in the short term – a by-product of the challenging economic climate.
“Traditional media is proven to drive awareness and builds an emotional connection which improves the perceptions of a brand,” Dubois says.
“The benefits of this are long-lived, but they can take time to eventuate as sales. Few marketers have this patience when their c-suite and shareholders are expecting results this quarter.”
Paling says while CEOs and CFOs should be educated to help counter short-termism in marketing, the real focus should be on marketers having the discussion.
“I think it’s much more likely that it’ll come effectively from marketers than from CFOs because marketers need to understand how marketing works,” Paling says.
“With the rise of digital, marketers have gone from having a reasonable understanding of how marketing works to having that understanding disrupted and challenged by new channels that don’t appear to behave in the same way as established channels.
“So now is the time where marketers need to have enough of an understanding of how both of those things work so that they can represent that story and not spend excessive amounts of money going after performance channels which are measured on something which gives you no idea of whether it’s actually delivering value or not.”
Nuemann sounds less optimistic that the industry will retire attribution modelling for alternatives.
“In advertising, if there’s something wrong we always think no one is harmed,” he says.
“But companies should learn from that.”
Have something to say on this? Share your views in the comments section below. Or if you have a news story or tip-off, drop us a line at adnews@yaffa.com.au
Sign up to the AdNews newsletter, like us on Facebook or follow us on Twitter for breaking stories and campaigns throughout the day.