The Validity of Impressions as a Media Dose Metric in a Tobacco Public Education Campaign Evaluation (Preprint)

Abstract

Evaluation research increasingly needs alternatives to target/gross rating points to comprehensively measure total exposure to modern multi-channel public education campaigns that use multiple channels, including TV, radio, digital video and paid social media, among others. Ratings data typically only captures delivery of broadcast media (TV and radio) and excludes other channels.

Studies are needed to validate cross-channel metrics such as impressions against self-reported exposure to campaign messages.

We analyzed over 3 years of ad impressions from the California Tobacco Prevention Program’s (CTPP) statewide tobacco education campaigns from August 2019 through December 2022. Impressions data varied across designated market areas (DMAs) and across time. These data were merged to individual respondents from 45 waves of online survey data of Californians ages 18-55 (n=151,649). Impressions were merged to respondents based on respondents’ DMAs and time of survey completion. We used logistic regression to estimate the odds of respondents’ campaign recall as a function of cumulative and past-3-month impressions delivered to each respondent’s DMA.

Cumulative impressions were positively and significantly associated with recall of each of the Flavors Hook Kids (OR = 1.15, p < 0.001), Dark Balloons/Apartment (OR = 1.20, p < 0.001), We Are Not Profit (OR = 1.36, p < 0.001), Tell Your Story (EVALI; OR = 1.06, p < 0.05), and Thrown Away/Little Big Lies (OR = 1.05, p < 0.01) campaigns. Impressions delivered in the past 3 months were associated with recall of the Flavors Hook Kids (OR = 1.13, p < 0.001), Dark Balloons/Apartment (OR = 1.08, p < 0.001), We Are Not Profit (OR = 1.14, p < 0.001), and Thrown Away/Little Big Lies (OR = 1.04, p < 0.001) campaigns. Past 3-month impressions were not significantly associated with Tell Your Story campaign recall. Overall, magnitudes of these associations were greater for cumulative impressions. We visualize recall based on post-estimation predicted values from our multivariate logistic regression models.

Variation in cumulative impressions for CTPP’s long-term multi-channel tobacco education campaign is predictive of increased self-reported campaign recall, suggesting that impressions may be a valid proxy for potential campaign exposure. The use of impressions for purposes of evaluating public education campaigns may help address current methodological limitations arising from the fragmented nature of modern multi-channel media campaigns.