Wharton’s Shawndra Hill discusses her research on TV ads and online search.
Shawndra Hill, a senior fellow at the Wharton Customer Analytics Initiative, likes to dig into the details. As someone who studies data mining, she looks for new ways to apply what she finds to solve business problems. Hill’s latest research paper, “Television and Digital Advertising: Second Screen Response and Coordination with Sponsored Search,” focuses on TV ads, online search and the connections between them. The paper was co-authored with Gordon Burtch from the University of Minnesota and Michael Barto, a data scientist at Microsoft. Hill recently spoke with [email protected] about what she found.
An edited transcript of the conversation follows.
[email protected]: What is the focus of this research?
Shawndra Hill: What we aim to achieve is to find new ways to measure TV ad effectiveness. Let me take a step back and talk about how people typically measure effectiveness. Large brand advertisers will usually ask another company to survey consumers and ask them questions like, “Did you see the ad? Would you like to recommend the product that was advertised to your friends? How did you feel about the ad?” So, questions about their attitudes.
They might also look at sales data and correlate that with the amount of spend that they’ve made. What we hope to do is look at more granular data that reveals itself in the searches people post on large search engines. What we’re hoping to do, or have done, is link TV ad data at the aggregate level, where it can tell us precisely which television show, what time and which locations an ad was shown. Then we look at search data around that TV ad, before and after, to see whether there was an impact on the search behavior.
We’re trying to look at the ability to coordinate advertising efforts, not just on television but also on digital platforms like sponsored searches. What we do is combine data from TV ads and then link that to the search data. Not just the searches, but also conditioned on somebody making a search — did they click on a sponsored search ad or not? We combined data from all of these sources to make causal claims about the impact of TV ads on digital behaviors, towards measuring the effectiveness of TV ads.
[email protected]: This research capitalizes on a phenomenon that’s grown in the last couple of years called second screening. No longer do we just watch TV, but we’re often sitting on the couch, looking at the TV and scrolling through our phones at the same time. When you looked at this a little more closely, what were some of the key takeaways that you found?
Hill: That’s a great observation. I should probably take a step back and tell you the research questions that we were interested in. The No. 1 research question we’re interested in is just that: How do behaviors in response to TV ads manifest themselves via these second screens? We do in fact see that there’s an increase in search behavior after a TV ad is shown. But that’s manifesting itself primarily on smartphones. The smaller the device, the more likely someone is to respond directly after a TV ad, digitally.
Because we have very granular-level search data, we were also interested in this interaction with the sponsored search ads. Finally, we wanted to look at how the TV ads impacted different users in various ways. For instance, we were interested in heterogenous effects on demographics — age and gender. Do certain genders respond differently to a particular creative that’s shown in a particular television show? Similarly, we looked at device. That’s how we were able to discern that the response was coming primarily from the mobile phones.
[email protected]: What were some of the findings that were most surprising to you? One thing that stood out to me is that you found when this increase in searching on your phone is going on, it only amounts to about three minutes.
“The smaller the device, the more likely someone is to respond directly after a TV ad, digitally.”
Hill: That’s right. You hit the nail on the head in terms of the surprising findings. There were two that I think are obvious in hindsight, but we didn’t necessarily anticipate. The first one was one that we already talked about. By disaggregating the data and looking at different cohorts — people searching from smartphones versus tablets versus PCs — we were able to see that the significant effect in terms of the bounce in searches after a TV ad was happening only on mobile phones. That’s the first thing that was surprising to us. Although in hindsight, it makes sense. If you’re sitting in front of the television, you’re not going to bring your desktop to watch television, right?
The second one was, because we’re looking at very fine-grain windows, we were able to see minute by minute the dynamic change in how people search after a TV ad. We found that we’re seeing it either in the first, second or third minute after the TV ad. At first, we thought, “Wow, we expected this thing to sort of slow down, but maybe tail off.” The reason, we suspect, is that TV ad segments are almost always exactly three minutes. So, people are probably switching their attention back to the television show after the TV ads are aired.
[email protected]: If I am an advertiser, then I have this three-minute window. People are on their phones, looking at these ads. What can I do to capitalize on this information that you found?
Hill: The implications of our work, I think, are many. The first one is that we’re finding that the search response to television ads is manifesting itself primarily on mobile phones, and from prior research — not ours — we know that people are more likely to click on the first ad [in a search listing] only on a mobile phone when compared to a PC or desktop. That’s primarily because of the footprint, right? You only see the first ad.
What that suggests is, if people really are moving to mobile phones when they’re watching television, then if you’re an advertiser and you really want to keep their attention, you should spend the money to make sure you’re the first ad that shows up.
But I think the work has even broader implications because we can see who is responding. Let’s take two examples. Let’s say you have only one ad creative — one TV ad, one commercial. Let’s say it’s a new product and you want to know who’s responding. You can launch that TV ad and basically look at the response in the way that we have and see which types of customers are responding and where. That can help you optimize your other advertising efforts, to do more here or less there, depending upon what you find.
The other example I wanted to point out is, if instead you have many ad creatives — maybe you’ve done some focus tests and know which one small groups like. But