Ironically, the recent proliferation of media communication channels has not made it simpler to reach more people. Instead, it has made it more difficult and time consuming to captivate audiences with the sheer number of outlets available. With traditional media channels, public health mass media campaigns have demonstrated the capacity to effectively influence adult health behaviours provided that campaigns employ a variety of communication tactics directed at changing the whole social system, thereby achieving high message reach and frequency with their intended audience.¹ Undoubtedly, large media campaigns that attempt to influence community-level factors aim to alter perceived social norms at the community level.² Although many of these campaigns have focused on reducing college student drinking norms^3–5 tobacco control media campaigns, in particular, have also provided rich sources of information for drawing these conclusions. A number of researchers have demonstrated that state-funded or nationally-funded counter-marketing media campaigns favourably impact youth smoking initiation.^6–17 It has been further shown that media campaigns can persuade adults by providing motivation to quit^18–25 and even reduce smoking prevalence.^26–28 Further information on the relative performance of different anti-tobacco advertising using naturalistic exposure in adults and youths can be found in chapter 11 of the National Cancer Institute's The Role of Media in Promoting and Reducing Tobacco Use.²⁹

For several decades, television advertisers outside of tobacco control have recognised the value of ad recall as a better measure of effectiveness than relying solely on more distal measures such as rating points, reach or frequency.³⁰ However, expenditure data estimated using price or media cost indices have been most widely adopted as the standard for evaluating generic advertising programmes because of their relative accessibility and ease of translation.³¹ Several studies have also examined the use of healthcare advertising exposure measured by advertisement expenditures or gross rating points (GRPs), which are essentially a measure of people who have seen an advertisement and its processing or communication effects.³² Others have examined downstream effects of exposure to advertising on target audience action, such as product requests or sales data.^{33, 34} However, much of the discussion in public health around advertising effectiveness and GRPs has taken place in the tobacco control arena.

Using GRPs or targeted rating points (TRPs), which are a measure of GRPs within a specific target population as a measure of exposure, anti-tobacco advertising has been shown to be associated with better smoking-related attitudes, beliefs and behaviours among youths³⁵ across gender and race/ethnic groups,³⁶ to generate calls to a telephone smoking cessation helpline,^{37, 38} to increase quitting behaviours among adults²³ and to decrease smoking prevalence among adults.³⁹ Most of these studies use short-term GRP levels, such as monthly GRPs, but some use cumulative GRPs as well.

Rather than examining distal behavioural measures of campaign effectiveness, in this study we assess efficiency of ad placement using ad recall as a proximal measure of effectiveness. This approach is comparable to analyses previously presented by Southwell and colleagues⁴⁰ and Niederdeppe.⁴¹ Southwell and colleagues demonstrated that a measure of ad recognition, termed ‘encoded’ ad exposure, for a national anti-drug media campaign is highly correlated with GRP density.⁴⁰ Niederdeppe provided evidence that both aided recall and confirmed ad awareness are positively correlated with cumulative GRPs, the sum of GRPs for each ad over the life of an ad, for the Florida Anti-Tobacco Media Evaluation surveys.⁴¹

As funding for tobacco control in California decreases,⁴² it has become even more critical to increase the efficiency of an ad reaching the media campaign's targeted audience. Therefore, the primary goal of the present analysis is to examine the relation between television advertisement awareness and the campaign's TRPs over the past decade with the goal of directing the programme's future marketing decisions.

From its inception in 1990 to the present, the intention of the California Tobacco Education Media Campaign has always been to increase public awareness around issues of secondhand smoke and the tobacco industry's deceptive marketing tactics, as well as promoting smoking cessation at the population level.⁴³ The California's Tobacco Control Program (CTCP) is somewhat distinctive by aiming its campaign messages at non-smokers, as much as smokers. Using paid commercials, the campaign incorporates several media components including television, radio, billboards, print and public relations activities.⁴³ The California media campaign has been evaluated in the past using a variety of macro-level approaches to show its effectiveness.^{6, 44–46}

In July 2001, CTCP began to monitor advertisement recall related to its media campaign using ongoing surveillance efforts. The first five series of the survey were conducted using telephone methodology by The Research Business International and Field and Tab Incorporation. The telephone survey used an aided recall method, where the interviewer provided a brief description of each advertisement and then asked the respondents if they had seen the ad.

In 2005, CTCP implemented population-based online panel methodology to conduct its media evaluation efforts using Knowledge Networks.⁴⁷ This prospective study design was intended to address causality issues (not presented in this paper). An aided recall method similar to the telephone survey was used to obtain ad recall.

Since most of the ads target smokers and non-smokers differently, resulting in different levels of aided recall by both groups, we believed that differences might exist in the relation between aided recall and TRP levels. Additionally, we hypothesised that the change in methodology from telephone to web may have impacted this relation (figure 1). Consequently, we examine each survey mode separately as well as combining both modes.

We first examined simple correlations between aided advertisement awareness and several measures of TRP including flight TRPs, cumulative TRPs and TRP density (table 4) by smoking status and overall. After looking at the distribution of the cumulative TRP measure with aided advertisement awareness, we determined that several large values influenced observed associations; hence, we also considered log-transformed cumulative TRP values.

Although cumulative TRPs correlated well with aided recall, the log-transformed cumulative TRP measure exhibited greater correlation across smoking status and survey mode. We observed higher correlations for the web survey than the telephone survey for most of the recall measures. Generally, correlations were higher among respondents who were non-smokers than those who were smokers regardless of survey mode.

As one might expect from the correlation analysis, the coefficient estimates of log-cumulative TRP in the non-smoker GEE models were larger than that in the smoker GEE models (table 5). Additionally, the estimates are larger from the web survey mode. A one-unit increase in log-cumulative TRPs led to a 13.6% increase in aided ad recall in all respondents (smokers and non-smokers) in the web surveys compared to a 5.3% increase in the telephone surveys. However owing to small sample sizes (ie, only 12 ads for the web-based survey and 17 ads for the telephone survey), these results should be considered cautiously.

All analyses were additionally conducted with unweighted data—that is, before weighting data to the California population, or using post-stratification weights. Largely, the results were similar to the results presented for the weighted data, except that overall results were skewed depending on the relative proportion of smokers in the sample.

Previous studies have suggested that cumulative campaign exposure measurements correlate strongly with television advertisement recall measures better than shorter-term campaign exposure measurements.⁴¹ Other researchers have examined relations with campaign exposure density or cumulative campaign exposure divided by the total number of weeks on the air.⁴⁰ Although Niederdeppe⁴¹ questions the relative validity of recall measures owing to low correlations with cumulative GRP measures in a public health setting, our findings suggest modestly higher correlations using cumulative TRPs.

Results from the correlation and GEE analyses suggest that aided recall is best predicted using the log transformation of cumulative TRPs in our data. Comparably, we found that only cumulative TRP levels were more highly associated with aided recall rather than flight TRP levels. This was true even though recall questions included a past 30-day or 60-day time restriction. These results were consistent for the telephone and web-based versions of the survey as well as for smokers and non-smokers. Although survey mode and smoking status appeared to impact our aided recall measure in a consistent way, their effects were not statistically different.

The natural shape of the log transformed data suggests that at some TRP level aided recall will reach a plateau. This finding implies that an advertisement may reach a point of diminishing returns that is commonly described as ‘wearout’. Although this term has been interpreted in a variety of ways, in this study, ‘wearout’ may imply that an ad does not have the capability of reaching new viewers. Put differently, an ad could still reinforce the message for an audience that has already seen the ad; however, the need for reinforcement would depend on whether CTCP would prefer to repeat the specific message in an advertisement.

We found a modest relation between TRP density and aided recall in our analysis. This finding along with our log transformed cumulative TRP results provides an argument for placing an advertisement at lower TRP levels over a longer period as comparable to having high TRP placement over a shorter period. In advertising literature, the practice of pulsing advertisements has been addressed in theoretical work but has focused on S-shaped sales demand or sales response to advertising.^56–60 This is an area that requires further investigation because the goals of public health advertising are different from typical product advertising.

Several limitations of this study exist. First, we take a macro-level approach to the data rather than looking at each individual respondent. This approach reduced our sample size to 29 data points. Second, the web-panel data are correlated over time as a portion of respondents who responded to multiple surveys over time. Similarly, data collected within the same wave are correlated because the same respondents answer ad recall questions for multiple ads within the same wave of data collection. To address this concern, we repeated our analyses using the wave of data collection as the repeated measure but found that the results were not meaningfully different. Third, we did not control for ad characteristics such as emotional reaction or theme that may have led to some higher or lower correlation values; hence, our estimates are valuable regardless of ad characteristics. Lastly, aided recall is not the ultimate measure of a media campaign's effectiveness. Still, aided recall is a useful measure that may help the media campaign address questions regarding efficiency of ad placement.

Further research is also needed on differences by mode of survey administration. We observed higher aided awareness using web-based surveys compared to telephone interviews. Partly this may be a consequence of having more time to answer questions online and the ability to use visual rather than auditory pathways to process the information.⁵²

To more fully understand the processes behind the longer-term impact of advertisement awareness on smoking behaviour, we intend to examine the web-based surveys from a longitudinal perspective. Each cross-sectional survey collection has provided the CTCP's media campaign with information about the effectiveness of specific advertisements that have played a part in programmatic decisions about which advertisements to place in future media flights and which types of advertisements to produce in the future. In light of declining funding, findings from this paper may offer the media campaign astute choices regarding future advertisement placement considering the important association of intensity and level of cumulative TRPs with aided ad recall.

Competing interests:None.

Contributors: All authors contributed to the conception and design of the study and the acquisition of data. DWC analyzed and interpreted the data. DWC and MVM drafted the article and revised it critically for important intellectual content. All authors had final approval of the version published.

Provenance and peer review: Not commissioned; externally peer reviewed.

The authors thank Sandra Soria and Teresa Kirkpatrick at the California Tobacco Control Program for help in obtaining and inputting data. A number of people have worked on and shaped this project over time, including Kristi Koumjian, Jessica Shumacher, Caroline Kurtz, Schaelene Rollins, Edie Ernst and Edana Fielden at the California Tobacco Control Program. We would also like to thank Anissa Bromley at Ground Zero, Deborah McCrudden at the Research Business International and Mike Dennis at Knowledge Networks for their assistance during data collection. Additionally, we are indebted to David Sly for his guidance along the way.